Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 934114 Details for
Bug 1136895
Import VmTemplate fails
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
Full engine log
engine.log (text/plain), 15.09 MB, created by
Vered Volansky
on 2014-09-03 14:30:42 UTC
(
hide
)
Description:
Full engine log
Filename:
MIME Type:
Creator:
Vered Volansky
Created:
2014-09-03 14:30:42 UTC
Size:
15.09 MB
patch
obsolete
>2014-08-10 13:14:38,262 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-10 13:14:38,263 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-10 13:14:38,264 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-10 13:14:38,265 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-10 13:14:38,266 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-10 13:14:38,267 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-10 13:14:38,268 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-10 13:14:38,269 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-10 13:14:38,269 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-10 13:14:38,270 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-10 13:14:38,270 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-10 13:14:38,271 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-10 13:14:38,272 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-10 13:14:38,272 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-10 13:14:38,273 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-10 13:14:38,273 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-10 13:14:38,274 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-10 13:14:38,274 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-10 13:14:38,275 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-10 13:14:38,275 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-10 13:14:38,276 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-10 13:14:38,276 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-10 13:14:38,277 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_USER" is "engine". >2014-08-10 13:14:38,278 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-10 13:14:38,278 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-10 13:14:38,279 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-10 13:14:38,280 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-10 13:14:38,281 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-10 13:14:38,281 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-10 13:14:38,282 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-10 13:14:38,282 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-10 13:14:38,283 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-10 13:14:38,283 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-10 13:14:38,284 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-10 13:14:38,284 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-10 13:14:38,285 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-10 13:14:38,285 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-10 13:14:38,289 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-10 13:14:38,290 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-10 13:14:38,291 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-10 13:14:38,291 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-10 13:14:38,292 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-10 13:14:38,292 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-10 13:14:38,293 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-10 13:14:38,293 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-10 13:14:38,295 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-10 13:14:38,296 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-10 13:14:38,297 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-10 13:14:38,301 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-10 13:14:38,302 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-10 13:14:38,302 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-10 13:14:38,303 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-10 13:14:38,303 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-10 13:14:38,304 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-10 13:14:38,311 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-10 13:14:38,312 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-10 13:14:38,312 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-10 13:14:38,313 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-10 13:14:38,313 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-10 13:14:38,314 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-10 13:14:38,314 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-10 13:14:38,315 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-10 13:14:38,316 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-10 13:14:38,316 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_USER" is "ovirt". >2014-08-10 13:14:38,316 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-10 13:14:38,317 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-10 13:14:38,318 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-10 13:14:38,318 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-10 13:14:38,318 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-10 13:14:38,319 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-10 13:14:38,654 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Start initializing Backend >2014-08-10 13:14:38,751 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:14:38,828 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:14:38,859 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-10 13:14:38,869 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-1) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-10 13:14:39,000 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Running ovirt-engine 3.5.0.0 >2014-08-10 13:14:39,001 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Start initializing dictionaries >2014-08-10 13:14:39,004 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Finished initializing dictionaries >2014-08-10 13:14:39,004 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing AuditLogCleanupManager >2014-08-10 13:14:39,005 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-10 13:14:39,021 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing AuditLogCleanupManager >2014-08-10 13:14:39,021 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing CommandEntityCleanupManager >2014-08-10 13:14:39,022 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-10 13:14:39,023 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing CommandEntityCleanupManager >2014-08-10 13:14:39,025 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Start initializing TagsDirector >2014-08-10 13:14:39,036 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Tag root added to tree >2014-08-10 13:14:39,045 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Finished initializing TagsDirector >2014-08-10 13:14:39,046 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Start initializing IsoDomainListSyncronizer >2014-08-10 13:14:39,053 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Finished initializing IsoDomainListSyncronizer >2014-08-10 13:14:39,112 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Completed initializing handlers >2014-08-10 13:14:39,132 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-08-10 13:14:39,141 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-10 13:14:39,143 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-08-10 13:14:39,143 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-08-10 13:14:39,144 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-08-10 13:14:39,145 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Mark incomplete jobs as UNKNOWN >2014-08-10 13:14:39,192 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Start initializing JobRepositoryCleanupManager >2014-08-10 13:14:39,193 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Finished initializing JobRepositoryCleanupManager >2014-08-10 13:14:39,193 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Start initializing AutoRecoveryManager >2014-08-10 13:14:39,196 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Finished initializing AutoRecoveryManager >2014-08-10 13:14:39,198 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Start initializing ExecutionMessageDirector >2014-08-10 13:14:39,202 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Finished initializing ExecutionMessageDirector >2014-08-10 13:14:39,251 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Start of enabled extensions list >2014-08-10 13:14:39,251 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-10 13:14:39,252 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-10 13:14:39,253 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) End of enabled extensions list >2014-08-10 13:14:39,322 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-8) Start initializing DbUserCacheManager >2014-08-10 13:14:39,324 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-8) Finished initializing DbUserCacheManager >2014-08-10 13:14:39,332 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-8) Initialization of AsyncTaskManager completed successfully. >2014-08-10 13:14:39,334 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-8) Start initializing ResourceManager >2014-08-10 13:14:39,415 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-8) Entered VdsManager constructor >2014-08-10 13:14:39,441 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-8) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-10 13:14:39,583 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-8) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-10 13:14:39,604 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-8) Finished initializing ResourceManager >2014-08-10 13:14:39,608 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-8) Initialization of OvfDataUpdater completed successfully. >2014-08-10 13:14:39,609 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) Start scheduling to enable vds load balancer >2014-08-10 13:14:39,612 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) Finished scheduling to enable vds load balancer >2014-08-10 13:14:39,612 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) Start HA Reservation check >2014-08-10 13:14:39,614 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) Finished HA Reservation check >2014-08-10 13:14:39,630 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-8) Init VM custom properties utilities >2014-08-10 13:14:39,635 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-8) Init device custom properties utilities >2014-08-10 13:14:39,639 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) Initializing Scheduling manager >2014-08-10 13:14:39,689 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-10 13:14:39,696 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-10 13:14:39,697 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-10 13:14:39,705 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) External scheduler disabled, discovery skipped >2014-08-10 13:14:39,706 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) Initialized Scheduling manager >2014-08-10 13:14:39,706 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-8) Initializing DWH Heart Beat >2014-08-10 13:14:39,710 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-8) DWH Heart Beat initialized >2014-08-10 13:14:39,768 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-10 13:14:43,324 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 73dea836 >2014-08-10 13:14:43,331 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 73dea836 >2014-08-10 13:14:43,348 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-10 13:14:43,357 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: UNASSIGNED not exist in string table >2014-08-10 13:14:43,357 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-10 13:14:43,358 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-10 13:14:43,359 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-10 13:14:43,359 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-10 13:14:43,361 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-10 13:14:43,361 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-10 13:14:43,362 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-10 13:14:43,363 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-10 13:14:43,368 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-10 13:14:43,369 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-10 13:14:43,370 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STOPPED not exist in string table >2014-08-10 13:14:43,371 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STARTED not exist in string table >2014-08-10 13:14:43,371 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_ERROR not exist in string table >2014-08-10 13:14:43,380 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-10 13:14:43,380 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-10 13:14:43,381 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-10 13:14:43,381 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-10 13:14:43,548 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-10 13:14:43,549 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is running with disabled SELinux. >2014-08-10 13:14:43,823 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h1 >2014-08-10 13:14:43,898 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [5c4b5f39] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-10 13:14:49,652 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-20) [6efa8e9f] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-10 13:14:49,668 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-20) [6efa8e9f] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-10 13:14:49,676 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-20) [6efa8e9f] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-10 13:14:49,743 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [6efa8e9f] Correlation ID: 6efa8e9f, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-10 13:14:49,809 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-20) [6efa8e9f] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-10 13:14:49,818 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-20) [6efa8e9f] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-10 13:14:49,842 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [6efa8e9f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-10 13:14:49,859 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [6efa8e9f] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 2dfc93ef >2014-08-10 13:14:49,933 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [6efa8e9f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 49 GB of free space >2014-08-10 13:14:49,958 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [6efa8e9f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 49 GB of free space >2014-08-10 13:14:49,959 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [6efa8e9f] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-10 13:14:49,967 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [6efa8e9f] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 2af1e871 >2014-08-10 13:14:49,974 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [6efa8e9f] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 2af1e871 >2014-08-10 13:14:49,974 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [6efa8e9f] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 2dfc93ef >2014-08-10 13:14:49,975 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [6efa8e9f] Discovered no tasks on Storage Pool Default >2014-08-10 13:15:39,641 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-10 13:15:39,643 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-10 13:15:39,644 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-10 13:15:39,645 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-10 13:15:39,646 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-10 13:15:39,647 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-10 13:15:39,648 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-10 13:15:39,649 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-10 13:15:39,649 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-10 13:15:39,650 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-10 13:15:39,650 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-10 13:15:39,651 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-10 13:15:39,651 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-10 13:15:39,652 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-10 13:15:39,653 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-10 13:15:39,653 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-10 13:15:39,654 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-10 13:15:39,654 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-10 13:15:39,655 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-10 13:15:39,655 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-10 13:15:39,656 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-10 13:15:39,656 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-10 13:15:39,657 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_USER" is "engine". >2014-08-10 13:15:39,657 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-10 13:15:39,658 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-10 13:15:39,659 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-10 13:15:39,659 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-10 13:15:39,660 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-10 13:15:39,661 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-10 13:15:39,662 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-10 13:15:39,662 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-10 13:15:39,663 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-10 13:15:39,663 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-10 13:15:39,664 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-10 13:15:39,664 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-10 13:15:39,665 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-10 13:15:39,665 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-10 13:15:39,666 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-10 13:15:39,667 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-10 13:15:39,667 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-10 13:15:39,668 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-10 13:15:39,668 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-10 13:15:39,669 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-10 13:15:39,669 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-10 13:15:39,670 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-10 13:15:39,670 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-10 13:15:39,671 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-10 13:15:39,671 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-10 13:15:39,672 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-10 13:15:39,672 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-10 13:15:39,673 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-10 13:15:39,674 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-10 13:15:39,674 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-10 13:15:39,675 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-10 13:15:39,675 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-10 13:15:39,676 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-10 13:15:39,676 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-10 13:15:39,677 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-10 13:15:39,678 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-10 13:15:39,678 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-10 13:15:39,678 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-10 13:15:39,679 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-10 13:15:39,680 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-10 13:15:39,680 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_USER" is "ovirt". >2014-08-10 13:15:39,681 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-10 13:15:39,681 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-10 13:15:39,683 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-10 13:15:39,686 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-10 13:15:39,686 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-10 13:15:39,687 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-10 13:15:40,058 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-8) Start initializing Backend >2014-08-10 13:15:40,184 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-8) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:15:40,296 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-8) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:15:40,337 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-8) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-10 13:15:40,350 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-8) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-10 13:15:40,493 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-8) Running ovirt-engine 3.5.0.0 >2014-08-10 13:15:40,494 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-8) Start initializing dictionaries >2014-08-10 13:15:40,497 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-8) Finished initializing dictionaries >2014-08-10 13:15:40,498 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-8) Start initializing AuditLogCleanupManager >2014-08-10 13:15:40,499 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-8) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-10 13:15:40,518 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-8) Finished initializing AuditLogCleanupManager >2014-08-10 13:15:40,518 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-8) Start initializing CommandEntityCleanupManager >2014-08-10 13:15:40,519 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-8) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-10 13:15:40,520 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-8) Finished initializing CommandEntityCleanupManager >2014-08-10 13:15:40,525 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-8) Start initializing TagsDirector >2014-08-10 13:15:40,536 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-8) Tag root added to tree >2014-08-10 13:15:40,544 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-8) Finished initializing TagsDirector >2014-08-10 13:15:40,545 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-8) Start initializing IsoDomainListSyncronizer >2014-08-10 13:15:40,553 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-8) Finished initializing IsoDomainListSyncronizer >2014-08-10 13:15:40,621 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-8) Completed initializing handlers >2014-08-10 13:15:40,641 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-8) Start initializing ErrorTranslatorImpl >2014-08-10 13:15:40,653 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-8) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-10 13:15:40,655 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-8) Finished initializing ErrorTranslatorImpl >2014-08-10 13:15:40,655 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-8) Start initializing ErrorTranslatorImpl >2014-08-10 13:15:40,656 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-8) Finished initializing ErrorTranslatorImpl >2014-08-10 13:15:40,657 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-8) Mark incomplete jobs as UNKNOWN >2014-08-10 13:15:40,682 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-8) Start initializing JobRepositoryCleanupManager >2014-08-10 13:15:40,683 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-8) Finished initializing JobRepositoryCleanupManager >2014-08-10 13:15:40,683 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-8) Start initializing AutoRecoveryManager >2014-08-10 13:15:40,685 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-8) Finished initializing AutoRecoveryManager >2014-08-10 13:15:40,686 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-8) Start initializing ExecutionMessageDirector >2014-08-10 13:15:40,688 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-8) Finished initializing ExecutionMessageDirector >2014-08-10 13:15:40,734 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-8) Start of enabled extensions list >2014-08-10 13:15:40,735 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-8) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-10 13:15:40,736 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-8) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-10 13:15:40,737 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-8) End of enabled extensions list >2014-08-10 13:15:40,794 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-7) Start initializing DbUserCacheManager >2014-08-10 13:15:40,795 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-7) Finished initializing DbUserCacheManager >2014-08-10 13:15:40,805 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-7) Initialization of AsyncTaskManager completed successfully. >2014-08-10 13:15:40,808 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-7) Start initializing ResourceManager >2014-08-10 13:15:40,824 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-10 13:15:40,888 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-7) Entered VdsManager constructor >2014-08-10 13:15:40,907 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-7) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-10 13:15:40,983 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-7) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-10 13:15:40,996 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-7) Finished initializing ResourceManager >2014-08-10 13:15:41,000 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-7) Initialization of OvfDataUpdater completed successfully. >2014-08-10 13:15:41,000 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) Start scheduling to enable vds load balancer >2014-08-10 13:15:41,001 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) Finished scheduling to enable vds load balancer >2014-08-10 13:15:41,001 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) Start HA Reservation check >2014-08-10 13:15:41,002 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) Finished HA Reservation check >2014-08-10 13:15:41,008 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-7) Init VM custom properties utilities >2014-08-10 13:15:41,009 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-7) Init device custom properties utilities >2014-08-10 13:15:41,014 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) Initializing Scheduling manager >2014-08-10 13:15:41,042 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) External scheduler disabled, discovery skipped >2014-08-10 13:15:41,042 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) Initialized Scheduling manager >2014-08-10 13:15:41,043 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-7) Initializing DWH Heart Beat >2014-08-10 13:15:41,044 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-7) DWH Heart Beat initialized >2014-08-10 13:15:41,048 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-10 13:15:41,050 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-10 13:15:41,050 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-10 13:15:44,623 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 21eeb0f >2014-08-10 13:15:44,633 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 21eeb0f >2014-08-10 13:15:44,645 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-10 13:15:44,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: UNASSIGNED not exist in string table >2014-08-10 13:15:44,652 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-10 13:15:44,652 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-10 13:15:44,653 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-10 13:15:44,653 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-10 13:15:44,654 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-10 13:15:44,655 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-10 13:15:44,656 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-10 13:15:44,657 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-10 13:15:44,658 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-10 13:15:44,659 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-10 13:15:44,659 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STOPPED not exist in string table >2014-08-10 13:15:44,660 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STARTED not exist in string table >2014-08-10 13:15:44,661 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_ERROR not exist in string table >2014-08-10 13:15:44,661 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-10 13:15:44,662 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-10 13:15:44,662 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-10 13:15:44,663 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-10 13:15:44,811 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-10 13:15:44,812 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is running with disabled SELinux. >2014-08-10 13:15:45,198 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h1 >2014-08-10 13:15:45,293 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [5569b6c0] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-10 13:15:51,039 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-19) [288657e1] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-10 13:15:51,048 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-19) [288657e1] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-10 13:15:51,053 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-19) [288657e1] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-10 13:15:51,114 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) [288657e1] Correlation ID: 288657e1, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-10 13:15:51,166 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-19) [288657e1] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-10 13:15:51,213 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-19) [288657e1] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-10 13:15:51,304 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) [288657e1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-10 13:15:51,324 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [288657e1] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 32947053 >2014-08-10 13:15:51,420 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) [288657e1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 49 GB of free space >2014-08-10 13:15:51,445 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) [288657e1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 49 GB of free space >2014-08-10 13:15:51,446 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [288657e1] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-10 13:15:51,456 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [288657e1] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 31d1e747 >2014-08-10 13:15:51,464 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [288657e1] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 31d1e747 >2014-08-10 13:15:51,465 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [288657e1] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 32947053 >2014-08-10 13:15:51,466 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [288657e1] Discovered no tasks on Storage Pool Default >2014-08-10 13:15:52,441 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-1) Running command: LoginAdminUserCommand internal: false. >2014-08-10 13:15:52,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-10 13:15:55,937 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-4) Running command: LoginUserCommand internal: false. >2014-08-10 13:16:02,125 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-1) [3b5aec92] Lock Acquired to object EngineLock [exclusiveLocks= key: de580b19-e975-426f-80de-6c7daf1f24ab value: VM >, sharedLocks= ] >2014-08-10 13:16:02,410 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] Running command: HibernateVmCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-10 13:16:02,460 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@afbeab9b), log id: 73d013e9 >2014-08-10 13:16:02,463 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] FINISH, UpdateVmDynamicDataVDSCommand, log id: 73d013e9 >2014-08-10 13:16:02,478 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = ef7c8ef2-cf66-4f60-a752-f1a79235e2d6, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = 952565e4-a91d-4876-a3b1-ccd5dfe2a8b0, newImageDescription = ), log id: 7278a6e9 >2014-08-10 13:16:02,490 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-10 13:16:02,937 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] FINISH, CreateImageVDSCommand, return: 952565e4-a91d-4876-a3b1-ccd5dfe2a8b0, log id: 7278a6e9 >2014-08-10 13:16:02,969 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 8ae74d40-2c2e-4226-9dd4-08ed57a8d4ad >2014-08-10 13:16:02,970 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] CommandMultiAsyncTasks::AttachTask: Attaching task 6af997dd-7966-4fa6-8a79-5e5e278f0ef4 to command 8ae74d40-2c2e-4226-9dd4-08ed57a8d4ad. >2014-08-10 13:16:02,990 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] Adding task 6af997dd-7966-4fa6-8a79-5e5e278f0ef4 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-10 13:16:03,016 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 4bd8a1f0-e234-40d6-a71c-9a93818794e4, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = bb223e58-a6b3-449a-a77a-344b4863850b, newImageDescription = ), log id: 3d7d67fa >2014-08-10 13:16:03,018 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-10 13:16:03,730 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] FINISH, CreateImageVDSCommand, return: bb223e58-a6b3-449a-a77a-344b4863850b, log id: 3d7d67fa >2014-08-10 13:16:03,839 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] CommandMultiAsyncTasks::AttachTask: Attaching task 5812d8ec-cc9a-412f-8005-99c507282276 to command 8ae74d40-2c2e-4226-9dd4-08ed57a8d4ad. >2014-08-10 13:16:03,869 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] Adding task 5812d8ec-cc9a-412f-8005-99c507282276 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-10 13:16:03,886 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@b6e8248c), log id: 420e3434 >2014-08-10 13:16:03,934 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] FINISH, UpdateVmDynamicDataVDSCommand, log id: 420e3434 >2014-08-10 13:16:04,017 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] Correlation ID: 3b5aec92, Job ID: ca864131-793a-4525-8d09-42afef54ed7d, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm4 was initiated by User admin (Host: h1). >2014-08-10 13:16:04,018 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] BaseAsyncTask::startPollingTask: Starting to poll task 6af997dd-7966-4fa6-8a79-5e5e278f0ef4. >2014-08-10 13:16:04,019 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] BaseAsyncTask::startPollingTask: Starting to poll task 5812d8ec-cc9a-412f-8005-99c507282276. >2014-08-10 13:16:04,020 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-31) [3b5aec92] Lock freed to object EngineLock [exclusiveLocks= key: de580b19-e975-426f-80de-6c7daf1f24ab value: VM >, sharedLocks= ] >2014-08-10 13:16:10,797 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-60) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-10 13:16:10,844 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-60) SPMAsyncTask::PollTask: Polling task 5812d8ec-cc9a-412f-8005-99c507282276 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-10 13:16:10,882 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-60) BaseAsyncTask::onTaskEndSuccess: Task 5812d8ec-cc9a-412f-8005-99c507282276 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-10 13:16:10,884 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-60) Task with DB Task ID 8cae746b-1044-4f73-92af-21405dd2e93b and VDSM Task ID 6af997dd-7966-4fa6-8a79-5e5e278f0ef4 is in state Polling. End action for command 8ae74d40-2c2e-4226-9dd4-08ed57a8d4ad will proceed when all the entitys tasks are completed. >2014-08-10 13:16:10,885 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-60) SPMAsyncTask::PollTask: Polling task 6af997dd-7966-4fa6-8a79-5e5e278f0ef4 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-10 13:16:10,899 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-60) BaseAsyncTask::onTaskEndSuccess: Task 6af997dd-7966-4fa6-8a79-5e5e278f0ef4 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-10 13:16:10,900 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-60) CommandAsyncTask::endActionIfNecessary: All tasks of command 8ae74d40-2c2e-4226-9dd4-08ed57a8d4ad has ended -> executing endAction >2014-08-10 13:16:10,901 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-60) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 8ae74d40-2c2e-4226-9dd4-08ed57a8d4ad): calling endAction . >2014-08-10 13:16:10,902 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-41) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-10 13:16:10,965 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-41) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-10 13:16:10,982 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-41) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,ef7c8ef2-cf66-4f60-a752-f1a79235e2d6,952565e4-a91d-4876-a3b1-ccd5dfe2a8b0,4bd8a1f0-e234-40d6-a71c-9a93818794e4,bb223e58-a6b3-449a-a77a-344b4863850b), log id: 497b0b5d >2014-08-10 13:16:10,984 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-41) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,ef7c8ef2-cf66-4f60-a752-f1a79235e2d6,952565e4-a91d-4876-a3b1-ccd5dfe2a8b0,4bd8a1f0-e234-40d6-a71c-9a93818794e4,bb223e58-a6b3-449a-a77a-344b4863850b), log id: 12fa0a51 >2014-08-10 13:16:11,000 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-41) FINISH, HibernateBrokerVDSCommand, log id: 12fa0a51 >2014-08-10 13:16:11,015 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-41) FINISH, HibernateVDSCommand, log id: 497b0b5d >2014-08-10 13:16:11,016 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-41) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-10 13:16:11,017 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-41) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-10 13:16:11,031 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-41) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 5812d8ec-cc9a-412f-8005-99c507282276 >2014-08-10 13:16:11,035 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-41) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 5812d8ec-cc9a-412f-8005-99c507282276), log id: 4e9ca165 >2014-08-10 13:16:11,037 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-41) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=5812d8ec-cc9a-412f-8005-99c507282276), log id: 36053c15 >2014-08-10 13:16:11,130 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-41) FINISH, HSMClearTaskVDSCommand, log id: 36053c15 >2014-08-10 13:16:11,130 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-41) FINISH, SPMClearTaskVDSCommand, log id: 4e9ca165 >2014-08-10 13:16:11,147 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-41) BaseAsyncTask::removeTaskFromDB: Removed task 5812d8ec-cc9a-412f-8005-99c507282276 from DataBase >2014-08-10 13:16:11,148 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-41) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 6af997dd-7966-4fa6-8a79-5e5e278f0ef4 >2014-08-10 13:16:11,148 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-41) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 6af997dd-7966-4fa6-8a79-5e5e278f0ef4), log id: 615680be >2014-08-10 13:16:11,150 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-41) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=6af997dd-7966-4fa6-8a79-5e5e278f0ef4), log id: 761e6bde >2014-08-10 13:16:11,223 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-41) FINISH, HSMClearTaskVDSCommand, log id: 761e6bde >2014-08-10 13:16:11,223 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-41) FINISH, SPMClearTaskVDSCommand, log id: 615680be >2014-08-10 13:16:11,230 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-41) BaseAsyncTask::removeTaskFromDB: Removed task 6af997dd-7966-4fa6-8a79-5e5e278f0ef4 from DataBase >2014-08-10 13:16:11,230 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-41) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 8ae74d40-2c2e-4226-9dd4-08ed57a8d4ad >2014-08-10 13:16:12,998 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) VM vm4 de580b19-e975-426f-80de-6c7daf1f24ab moved from SavingState --> Up >2014-08-10 13:17:40,799 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-79) Setting new tasks map. The map contains now 0 tasks >2014-08-10 13:17:40,800 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-79) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-10 13:41:43,696 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8080-1) [5bcaa26f] Failed to initiate vdsm-id request on host with message Command returned failure code 127 during SSH session 'root@galactica.tlv.redhat.com' >2014-08-10 13:41:43,718 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8080-1) [5bcaa26f] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-00000000039f Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-08-10 13:41:43,753 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (http--0.0.0.0-8080-1) [e68096c] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 0472f9c2-dacf-4393-9fc3-5840ef369e1f Type: VDS >2014-08-10 13:41:43,785 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (http--0.0.0.0-8080-1) [e68096c] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-00000000037f value: REGISTER_VDS >, sharedLocks= ] >2014-08-10 13:41:43,788 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (http--0.0.0.0-8080-1) [e68096c] START, RemoveVdsVDSCommand(HostName = h2, HostId = 0472f9c2-dacf-4393-9fc3-5840ef369e1f), log id: 66fa7005 >2014-08-10 13:41:43,789 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (http--0.0.0.0-8080-1) [e68096c] FINISH, RemoveVdsVDSCommand, log id: 66fa7005 >2014-08-10 13:41:43,790 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [e68096c] START, AddVdsVDSCommand(HostName = h2, HostId = 0472f9c2-dacf-4393-9fc3-5840ef369e1f), log id: 1d94d886 >2014-08-10 13:41:43,791 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [e68096c] AddVds - entered , starting logic to add VDS 0472f9c2-dacf-4393-9fc3-5840ef369e1f >2014-08-10 13:41:43,792 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [e68096c] AddVds - VDS 0472f9c2-dacf-4393-9fc3-5840ef369e1f was added, will try to add it to the resource manager >2014-08-10 13:41:43,793 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (http--0.0.0.0-8080-1) [e68096c] Entered VdsManager constructor >2014-08-10 13:41:43,804 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (http--0.0.0.0-8080-1) [e68096c] Initialize vdsBroker (galactica.tlv.redhat.com,54,321) >2014-08-10 13:41:43,835 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (http--0.0.0.0-8080-1) [e68096c] VDS 0472f9c2-dacf-4393-9fc3-5840ef369e1f was added to the Resource Manager >2014-08-10 13:41:43,836 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [e68096c] FINISH, AddVdsVDSCommand, log id: 1d94d886 >2014-08-10 13:41:43,840 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [e68096c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h2. >2014-08-10 13:41:43,870 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Lock Acquired to object EngineLock [exclusiveLocks= key: 0472f9c2-dacf-4393-9fc3-5840ef369e1f value: VDS >, sharedLocks= ] >2014-08-10 13:41:43,876 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [e68096c] Correlation ID: 5bcaa26f, Job ID: 883a39a1-f2d4-4989-a14f-c5224f128730, Call Stack: null, Custom Event ID: -1, Message: Host h2 was added by admin. >2014-08-10 13:41:43,901 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 0472f9c2-dacf-4393-9fc3-5840ef369e1f Type: VDS >2014-08-10 13:41:43,906 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Before Installation host 0472f9c2-dacf-4393-9fc3-5840ef369e1f, h2 >2014-08-10 13:41:43,910 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 0472f9c2-dacf-4393-9fc3-5840ef369e1f, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 5aa57f9d >2014-08-10 13:41:43,917 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] FINISH, SetVdsStatusVDSCommand, log id: 5aa57f9d >2014-08-10 13:41:43,955 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Installation galactica.tlv.redhat.com: Connected to host galactica.tlv.redhat.com with SSH key fingerprint: af:6b:19:0c:de:1c:76:eb:a5:97:b3:fa:05:62:84:04 >2014-08-10 13:41:43,975 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Correlation ID: 450fbc5c, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Connected to host galactica.tlv.redhat.com with SSH key fingerprint: af:6b:19:0c:de:1c:76:eb:a5:97:b3:fa:05:62:84:04. >2014-08-10 13:41:44,035 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Installation of galactica.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /home/vvolansk/ovirt-engine/var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-08-10 13:41:44,183 INFO [org.ovirt.engine.core.utils.archivers.tar.CachedTar] (org.ovirt.thread.pool-8-thread-1) Tarball '/home/vvolansk/ovirt-engine/var/cache/ovirt-engine/ovirt-host-deploy.tar' is out of date, re-creating >2014-08-10 13:41:45,118 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-8-thread-1) SSH execute root@galactica.tlv.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-08-10 13:41:45,143 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Error during deploy dialog: java.io.IOException: Unexpected connection termination > at org.ovirt.otopi.dialog.MachineDialogParser.nextEvent(MachineDialogParser.java:388) [otopi.jar:] > at org.ovirt.otopi.dialog.MachineDialogParser.nextEvent(MachineDialogParser.java:405) [otopi.jar:] > at org.ovirt.engine.core.bll.VdsDeploy._threadMain(VdsDeploy.java:818) [bll.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.access$2000(VdsDeploy.java:83) [bll.jar:] > at org.ovirt.engine.core.bll.VdsDeploy$51.run(VdsDeploy.java:966) [bll.jar:] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:41:45,144 ERROR [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-8-thread-1) Swallowing exception as preferring stderr: java.io.IOException: Command returned failure code 127 during SSH session 'root@galactica.tlv.redhat.com' > at org.ovirt.engine.core.uutils.ssh.SSHClient.executeCommand(SSHClient.java:527) [uutils.jar:] > at org.ovirt.engine.core.uutils.ssh.SSHDialog.executeCommand(SSHDialog.java:318) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1115) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2239) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:41:45,184 ERROR [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-8-thread-1) SSH error running command root@galactica.tlv.redhat.com:'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True': java.lang.RuntimeException: Unexpected error during execution: bash: tar: command not found > > at org.ovirt.engine.core.uutils.ssh.SSHDialog.executeCommand(SSHDialog.java:337) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1115) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2239) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:41:45,204 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Error during host galactica.tlv.redhat.com install: java.lang.RuntimeException: Unexpected error during execution: bash: tar: command not found > > at org.ovirt.engine.core.uutils.ssh.SSHDialog.executeCommand(SSHDialog.java:337) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1115) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2239) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:41:45,219 ERROR [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Installation galactica.tlv.redhat.com: Unexpected error during execution: bash: tar: command not found > >2014-08-10 13:41:45,242 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Correlation ID: 450fbc5c, Call Stack: null, Custom Event ID: -1, Message: Failed to install Host h2. Unexpected error during execution: bash: tar: command not found >. >2014-08-10 13:41:45,243 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Error during host galactica.tlv.redhat.com install, prefering first exception: java.io.IOException: Unexpected connection termination > at org.ovirt.otopi.dialog.MachineDialogParser.nextEvent(MachineDialogParser.java:388) [otopi.jar:] > at org.ovirt.otopi.dialog.MachineDialogParser.nextEvent(MachineDialogParser.java:405) [otopi.jar:] > at org.ovirt.engine.core.bll.VdsDeploy._threadMain(VdsDeploy.java:818) [bll.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.access$2000(VdsDeploy.java:83) [bll.jar:] > at org.ovirt.engine.core.bll.VdsDeploy$51.run(VdsDeploy.java:966) [bll.jar:] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:41:45,244 ERROR [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Host installation failed for host 0472f9c2-dacf-4393-9fc3-5840ef369e1f, h2.: java.io.IOException: Unexpected connection termination > at org.ovirt.otopi.dialog.MachineDialogParser.nextEvent(MachineDialogParser.java:388) [otopi.jar:] > at org.ovirt.otopi.dialog.MachineDialogParser.nextEvent(MachineDialogParser.java:405) [otopi.jar:] > at org.ovirt.engine.core.bll.VdsDeploy._threadMain(VdsDeploy.java:818) [bll.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.access$2000(VdsDeploy.java:83) [bll.jar:] > at org.ovirt.engine.core.bll.VdsDeploy$51.run(VdsDeploy.java:966) [bll.jar:] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:41:45,247 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 0472f9c2-dacf-4393-9fc3-5840ef369e1f, status=InstallFailed, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 24a545a1 >2014-08-10 13:41:45,259 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] FINISH, SetVdsStatusVDSCommand, log id: 24a545a1 >2014-08-10 13:41:45,275 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Correlation ID: 450fbc5c, Job ID: 883a39a1-f2d4-4989-a14f-c5224f128730, Call Stack: null, Custom Event ID: -1, Message: Host h2 installation failed. Unexpected connection termination. >2014-08-10 13:41:45,284 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-1) [450fbc5c] Lock freed to object EngineLock [exclusiveLocks= key: 0472f9c2-dacf-4393-9fc3-5840ef369e1f value: VDS >, sharedLocks= ] >2014-08-10 13:41:46,836 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Initializing Host: h2 >2014-08-10 13:42:00,698 WARN [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (http--0.0.0.0-8080-1) [4c70eca8] CanDoAction of action MaintenanceNumberOfVdss failed. Reasons:VAR__TYPE__HOST,VAR__ACTION__MAINTENANCE,VDS_CANNOT_MAINTENANCE_VDS_IS_NOT_OPERATIONAL >2014-08-10 13:42:08,678 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (http--0.0.0.0-8080-1) [3404c93e] Lock Acquired to object EngineLock [exclusiveLocks= key: 0472f9c2-dacf-4393-9fc3-5840ef369e1f value: VDS >, sharedLocks= ] >2014-08-10 13:42:08,686 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-8-thread-38) [3404c93e] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 0472f9c2-dacf-4393-9fc3-5840ef369e1f Type: VDSAction group DELETE_HOST with role type ADMIN >2014-08-10 13:42:08,815 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-8-thread-38) [3404c93e] START, RemoveVdsVDSCommand( HostId = 0472f9c2-dacf-4393-9fc3-5840ef369e1f), log id: 20169f4c >2014-08-10 13:42:08,815 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-8-thread-38) [3404c93e] vdsManager::disposing >2014-08-10 13:42:08,821 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (org.ovirt.thread.pool-8-thread-38) Message sent: UNSUBSCRIBE >id:null > > >2014-08-10 13:42:08,821 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (org.ovirt.thread.pool-8-thread-38) Message sent: DISCONNECT >receipt:cbb14eb7-69da-4b45-a872-d91ea2f59a68 > > >2014-08-10 13:42:08,822 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-8-thread-38) [3404c93e] FINISH, RemoveVdsVDSCommand, log id: 20169f4c >2014-08-10 13:42:08,830 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-38) [3404c93e] Correlation ID: 3404c93e, Call Stack: null, Custom Event ID: -1, Message: Host h2 was removed by admin. >2014-08-10 13:42:08,831 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-8-thread-38) [3404c93e] Lock freed to object EngineLock [exclusiveLocks= key: 0472f9c2-dacf-4393-9fc3-5840ef369e1f value: VDS >, sharedLocks= ] >2014-08-10 13:45:13,237 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8080-1) [65960527] Failed to initiate vdsm-id request on host with message Command returned failure code 127 during SSH session 'root@galactica.tlv.redhat.com' >2014-08-10 13:45:13,264 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8080-1) [65960527] Running command: AddVdsCommand internal: false. Entities affected : ID: 8fc43cd8-abe9-4ee7-834b-6416aed91c63 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-08-10 13:45:13,282 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (http--0.0.0.0-8080-1) [2c3ef0fd] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 7140a009-99a5-4aa6-b769-4299ed51b873 Type: VDS >2014-08-10 13:45:13,289 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (http--0.0.0.0-8080-1) [2c3ef0fd] Lock freed to object EngineLock [exclusiveLocks= key: 76bcbfe3-3419-4fdb-9b89-d147b7fdef99 value: REGISTER_VDS >, sharedLocks= ] >2014-08-10 13:45:13,291 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (http--0.0.0.0-8080-1) [2c3ef0fd] START, RemoveVdsVDSCommand(HostName = h2, HostId = 7140a009-99a5-4aa6-b769-4299ed51b873), log id: 14441339 >2014-08-10 13:45:13,291 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (http--0.0.0.0-8080-1) [2c3ef0fd] FINISH, RemoveVdsVDSCommand, log id: 14441339 >2014-08-10 13:45:13,292 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [2c3ef0fd] START, AddVdsVDSCommand(HostName = h2, HostId = 7140a009-99a5-4aa6-b769-4299ed51b873), log id: eaa5640 >2014-08-10 13:45:13,293 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [2c3ef0fd] AddVds - entered , starting logic to add VDS 7140a009-99a5-4aa6-b769-4299ed51b873 >2014-08-10 13:45:13,294 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [2c3ef0fd] AddVds - VDS 7140a009-99a5-4aa6-b769-4299ed51b873 was added, will try to add it to the resource manager >2014-08-10 13:45:13,295 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (http--0.0.0.0-8080-1) [2c3ef0fd] Entered VdsManager constructor >2014-08-10 13:45:13,301 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (http--0.0.0.0-8080-1) [2c3ef0fd] Initialize vdsBroker (galactica.tlv.redhat.com,54,321) >2014-08-10 13:45:13,301 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (http--0.0.0.0-8080-1) [2c3ef0fd] VDS 7140a009-99a5-4aa6-b769-4299ed51b873 was added to the Resource Manager >2014-08-10 13:45:13,302 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [2c3ef0fd] FINISH, AddVdsVDSCommand, log id: eaa5640 >2014-08-10 13:45:13,304 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [2c3ef0fd] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h2. >2014-08-10 13:45:13,330 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [2c3ef0fd] Correlation ID: 65960527, Job ID: 44baca53-7555-49cf-ae02-d16d1719d1cb, Call Stack: null, Custom Event ID: -1, Message: Host h2 was added by admin. >2014-08-10 13:45:13,339 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Lock Acquired to object EngineLock [exclusiveLocks= key: 7140a009-99a5-4aa6-b769-4299ed51b873 value: VDS >, sharedLocks= ] >2014-08-10 13:45:13,355 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 7140a009-99a5-4aa6-b769-4299ed51b873 Type: VDS >2014-08-10 13:45:13,356 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Before Installation host 7140a009-99a5-4aa6-b769-4299ed51b873, h2 >2014-08-10 13:45:13,359 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-22) [73e3795a] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 7140a009-99a5-4aa6-b769-4299ed51b873, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 212e6117 >2014-08-10 13:45:13,372 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-22) [73e3795a] FINISH, SetVdsStatusVDSCommand, log id: 212e6117 >2014-08-10 13:45:13,402 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Installation galactica.tlv.redhat.com: Connected to host galactica.tlv.redhat.com with SSH key fingerprint: af:6b:19:0c:de:1c:76:eb:a5:97:b3:fa:05:62:84:04 >2014-08-10 13:45:13,430 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Correlation ID: 73e3795a, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Connected to host galactica.tlv.redhat.com with SSH key fingerprint: af:6b:19:0c:de:1c:76:eb:a5:97:b3:fa:05:62:84:04. >2014-08-10 13:45:13,482 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Installation of galactica.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /home/vvolansk/ovirt-engine/var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-08-10 13:45:13,488 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-8-thread-22) SSH execute root@galactica.tlv.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-08-10 13:45:13,510 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Error during deploy dialog: java.io.IOException: Unexpected connection termination > at org.ovirt.otopi.dialog.MachineDialogParser.nextEvent(MachineDialogParser.java:388) [otopi.jar:] > at org.ovirt.otopi.dialog.MachineDialogParser.nextEvent(MachineDialogParser.java:405) [otopi.jar:] > at org.ovirt.engine.core.bll.VdsDeploy._threadMain(VdsDeploy.java:818) [bll.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.access$2000(VdsDeploy.java:83) [bll.jar:] > at org.ovirt.engine.core.bll.VdsDeploy$51.run(VdsDeploy.java:966) [bll.jar:] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:45:13,512 ERROR [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-8-thread-22) Swallowing exception as preferring stderr: java.io.IOException: Command returned failure code 127 during SSH session 'root@galactica.tlv.redhat.com' > at org.ovirt.engine.core.uutils.ssh.SSHClient.executeCommand(SSHClient.java:527) [uutils.jar:] > at org.ovirt.engine.core.uutils.ssh.SSHDialog.executeCommand(SSHDialog.java:318) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1115) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2239) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:45:13,530 ERROR [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-8-thread-22) SSH error running command root@galactica.tlv.redhat.com:'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True': java.lang.RuntimeException: Unexpected error during execution: bash: tar: command not found > > at org.ovirt.engine.core.uutils.ssh.SSHDialog.executeCommand(SSHDialog.java:337) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1115) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2239) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:45:13,549 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Error during host galactica.tlv.redhat.com install: java.lang.RuntimeException: Unexpected error during execution: bash: tar: command not found > > at org.ovirt.engine.core.uutils.ssh.SSHDialog.executeCommand(SSHDialog.java:337) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1115) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2239) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:45:13,567 ERROR [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Installation galactica.tlv.redhat.com: Unexpected error during execution: bash: tar: command not found > >2014-08-10 13:45:13,579 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Correlation ID: 73e3795a, Call Stack: null, Custom Event ID: -1, Message: Failed to install Host h2. Unexpected error during execution: bash: tar: command not found >. >2014-08-10 13:45:13,579 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Error during host galactica.tlv.redhat.com install, prefering first exception: java.io.IOException: Unexpected connection termination > at org.ovirt.otopi.dialog.MachineDialogParser.nextEvent(MachineDialogParser.java:388) [otopi.jar:] > at org.ovirt.otopi.dialog.MachineDialogParser.nextEvent(MachineDialogParser.java:405) [otopi.jar:] > at org.ovirt.engine.core.bll.VdsDeploy._threadMain(VdsDeploy.java:818) [bll.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.access$2000(VdsDeploy.java:83) [bll.jar:] > at org.ovirt.engine.core.bll.VdsDeploy$51.run(VdsDeploy.java:966) [bll.jar:] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:45:13,581 ERROR [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Host installation failed for host 7140a009-99a5-4aa6-b769-4299ed51b873, h2.: java.io.IOException: Unexpected connection termination > at org.ovirt.otopi.dialog.MachineDialogParser.nextEvent(MachineDialogParser.java:388) [otopi.jar:] > at org.ovirt.otopi.dialog.MachineDialogParser.nextEvent(MachineDialogParser.java:405) [otopi.jar:] > at org.ovirt.engine.core.bll.VdsDeploy._threadMain(VdsDeploy.java:818) [bll.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.access$2000(VdsDeploy.java:83) [bll.jar:] > at org.ovirt.engine.core.bll.VdsDeploy$51.run(VdsDeploy.java:966) [bll.jar:] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 13:45:13,583 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-22) [73e3795a] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 7140a009-99a5-4aa6-b769-4299ed51b873, status=InstallFailed, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 1ab334c1 >2014-08-10 13:45:13,595 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-22) [73e3795a] FINISH, SetVdsStatusVDSCommand, log id: 1ab334c1 >2014-08-10 13:45:13,603 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Correlation ID: 73e3795a, Job ID: 44baca53-7555-49cf-ae02-d16d1719d1cb, Call Stack: null, Custom Event ID: -1, Message: Host h2 installation failed. Unexpected connection termination. >2014-08-10 13:45:13,612 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-22) [73e3795a] Lock freed to object EngineLock [exclusiveLocks= key: 7140a009-99a5-4aa6-b769-4299ed51b873 value: VDS >, sharedLocks= ] >2014-08-10 13:45:16,303 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Initializing Host: h2 >2014-08-10 13:46:04,876 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (http--0.0.0.0-8080-1) [11b8aae7] Lock Acquired to object EngineLock [exclusiveLocks= key: 7140a009-99a5-4aa6-b769-4299ed51b873 value: VDS >, sharedLocks= ] >2014-08-10 13:46:04,881 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-8-thread-34) [11b8aae7] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 7140a009-99a5-4aa6-b769-4299ed51b873 Type: VDSAction group DELETE_HOST with role type ADMIN >2014-08-10 13:46:04,993 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-8-thread-34) [11b8aae7] START, RemoveVdsVDSCommand( HostId = 7140a009-99a5-4aa6-b769-4299ed51b873), log id: 3ad35eef >2014-08-10 13:46:04,993 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-8-thread-34) [11b8aae7] vdsManager::disposing >2014-08-10 13:46:04,994 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-8-thread-34) [11b8aae7] FINISH, RemoveVdsVDSCommand, log id: 3ad35eef >2014-08-10 13:46:05,000 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-34) [11b8aae7] Correlation ID: 11b8aae7, Call Stack: null, Custom Event ID: -1, Message: Host h2 was removed by admin. >2014-08-10 13:46:05,001 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-8-thread-34) [11b8aae7] Lock freed to object EngineLock [exclusiveLocks= key: 7140a009-99a5-4aa6-b769-4299ed51b873 value: VDS >, sharedLocks= ] >2014-08-10 14:04:37,405 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8080-1) [4a4d98a5] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server galactica.tlv.redhat.com,VDS_CANNOT_INSTALL_EMPTY_PASSWORD >2014-08-10 14:04:44,713 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8080-1) [1975f406] Failed to initiate vdsm-id request on host with message Command returned failure code 127 during SSH session 'root@galactica.tlv.redhat.com' >2014-08-10 14:04:44,770 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8080-1) [1975f406] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-00000000039f Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-08-10 14:04:44,789 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (http--0.0.0.0-8080-1) [63bfbc9d] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDS >2014-08-10 14:04:44,804 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (http--0.0.0.0-8080-1) [63bfbc9d] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-00000000037f value: REGISTER_VDS >, sharedLocks= ] >2014-08-10 14:04:44,806 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (http--0.0.0.0-8080-1) [63bfbc9d] START, RemoveVdsVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314), log id: 7973325e >2014-08-10 14:04:44,807 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (http--0.0.0.0-8080-1) [63bfbc9d] FINISH, RemoveVdsVDSCommand, log id: 7973325e >2014-08-10 14:04:44,808 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [63bfbc9d] START, AddVdsVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314), log id: 623324b5 >2014-08-10 14:04:44,808 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [63bfbc9d] AddVds - entered , starting logic to add VDS 70abf755-e0dc-4d23-9bd7-a40bc21c4314 >2014-08-10 14:04:44,809 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [63bfbc9d] AddVds - VDS 70abf755-e0dc-4d23-9bd7-a40bc21c4314 was added, will try to add it to the resource manager >2014-08-10 14:04:44,810 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (http--0.0.0.0-8080-1) [63bfbc9d] Entered VdsManager constructor >2014-08-10 14:04:44,819 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (http--0.0.0.0-8080-1) [63bfbc9d] Initialize vdsBroker (galactica.tlv.redhat.com,54,321) >2014-08-10 14:04:44,820 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (http--0.0.0.0-8080-1) [63bfbc9d] VDS 70abf755-e0dc-4d23-9bd7-a40bc21c4314 was added to the Resource Manager >2014-08-10 14:04:44,820 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [63bfbc9d] FINISH, AddVdsVDSCommand, log id: 623324b5 >2014-08-10 14:04:44,825 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [63bfbc9d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h2. >2014-08-10 14:04:44,854 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [63bfbc9d] Correlation ID: 1975f406, Job ID: fad1c0e2-8080-496c-8358-5105e69ac9a7, Call Stack: null, Custom Event ID: -1, Message: Host h2 was added by admin. >2014-08-10 14:04:44,862 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Lock Acquired to object EngineLock [exclusiveLocks= key: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 value: VDS >, sharedLocks= ] >2014-08-10 14:04:44,879 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDS >2014-08-10 14:04:44,879 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Before Installation host 70abf755-e0dc-4d23-9bd7-a40bc21c4314, h2 >2014-08-10 14:04:44,881 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 760f7ca4 >2014-08-10 14:04:44,887 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] FINISH, SetVdsStatusVDSCommand, log id: 760f7ca4 >2014-08-10 14:04:44,933 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Installation galactica.tlv.redhat.com: Connected to host galactica.tlv.redhat.com with SSH key fingerprint: af:6b:19:0c:de:1c:76:eb:a5:97:b3:fa:05:62:84:04 >2014-08-10 14:04:44,945 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Connected to host galactica.tlv.redhat.com with SSH key fingerprint: af:6b:19:0c:de:1c:76:eb:a5:97:b3:fa:05:62:84:04. >2014-08-10 14:04:45,011 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Installation of galactica.tlv.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /home/vvolansk/ovirt-engine/var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-08-10 14:04:45,017 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-8-thread-12) SSH execute root@galactica.tlv.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-08-10 14:04:45,429 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Initializing >2014-08-10 14:04:45,475 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Initializing. >2014-08-10 14:04:45,494 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Environment setup >2014-08-10 14:04:45,508 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Environment setup. >2014-08-10 14:04:45,518 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Environment packages setup >2014-08-10 14:04:45,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Environment packages setup. >2014-08-10 14:04:47,821 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Initializing Host: h2 >2014-08-10 14:04:51,176 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Status: Downloading Packages >2014-08-10 14:04:51,239 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Status: Downloading Packages. >2014-08-10 14:04:51,240 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:dmidecode-2.12-4.fc20.x86_64 >2014-08-10 14:04:51,248 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:dmidecode-2.12-4.fc20.x86_64. >2014-08-10 14:04:51,248 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Status: Check Package Signatures >2014-08-10 14:04:51,256 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Status: Check Package Signatures. >2014-08-10 14:04:51,256 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Status: Running Test Transaction >2014-08-10 14:04:51,264 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Status: Running Test Transaction. >2014-08-10 14:04:51,265 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Status: Running Transaction >2014-08-10 14:04:51,272 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Status: Running Transaction. >2014-08-10 14:04:51,335 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 1/1: 1:dmidecode-2.12-4.fc20.x86_64 >2014-08-10 14:04:51,347 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 1/1: 1:dmidecode-2.12-4.fc20.x86_64. >2014-08-10 14:04:51,664 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 1/1: dmidecode.x86_64 1:2.12-4.fc20 - u >2014-08-10 14:04:51,670 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 1/1: dmidecode.x86_64 1:2.12-4.fc20 - u. >2014-08-10 14:04:52,044 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Programs detection >2014-08-10 14:04:52,051 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Programs detection. >2014-08-10 14:04:52,059 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Environment customization >2014-08-10 14:04:52,067 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Environment customization. >2014-08-10 14:04:52,068 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Kdump unsupported >2014-08-10 14:04:52,076 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Kdump unsupported. >2014-08-10 14:04:52,085 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20140810223807-pvn2pz.log' >2014-08-10 14:04:52,092 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Logs at host located at: '/tmp/ovirt-host-deploy-20140810223807-pvn2pz.log'. >2014-08-10 14:04:52,124 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host galactica.tlv.redhat.com reports unique id 4C4C4544-0047-3910-8046-CAC04F46354A >2014-08-10 14:04:52,139 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0047-3910-8046-CAC04F46354A to Host galactica.tlv.redhat.com >2014-08-10 14:04:52,201 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Disabling Kdump integration >2014-08-10 14:04:52,208 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Disabling Kdump integration. >2014-08-10 14:04:52,226 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Setup validation >2014-08-10 14:04:52,242 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Setup validation. >2014-08-10 14:04:52,302 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Hardware supports virtualization >2014-08-10 14:04:52,316 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Hardware supports virtualization. >2014-08-10 14:04:52,651 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Transaction setup >2014-08-10 14:04:52,663 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Transaction setup. >2014-08-10 14:04:52,789 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Misc configuration >2014-08-10 14:04:52,804 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Misc configuration. >2014-08-10 14:04:52,805 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Package installation >2014-08-10 14:04:52,854 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Package installation. >2014-08-10 14:04:58,436 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Status: Downloading Packages >2014-08-10 14:04:58,469 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Status: Downloading Packages. >2014-08-10 14:04:58,496 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: OpenIPMI-modalias-2.0.19-9.fc20.x86_64 >2014-08-10 14:04:58,511 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: OpenIPMI-modalias-2.0.19-9.fc20.x86_64. >2014-08-10 14:04:58,511 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: SDL-1.2.15-13.fc20.x86_64 >2014-08-10 14:04:58,519 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: SDL-1.2.15-13.fc20.x86_64. >2014-08-10 14:04:58,524 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: alsa-lib-1.0.27.2-2.fc20.x86_64 >2014-08-10 14:04:58,535 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: alsa-lib-1.0.27.2-2.fc20.x86_64. >2014-08-10 14:04:58,536 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: amtterm-1.3-7.fc20.x86_64 >2014-08-10 14:04:58,585 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: amtterm-1.3-7.fc20.x86_64. >2014-08-10 14:04:58,586 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: at-spi2-atk-2.10.2-1.fc20.x86_64 >2014-08-10 14:04:58,593 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: at-spi2-atk-2.10.2-1.fc20.x86_64. >2014-08-10 14:04:58,594 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: at-spi2-core-2.10.2-1.fc20.x86_64 >2014-08-10 14:04:58,602 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: at-spi2-core-2.10.2-1.fc20.x86_64. >2014-08-10 14:04:58,602 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: atk-2.10.0-1.fc20.x86_64 >2014-08-10 14:04:58,610 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: atk-2.10.0-1.fc20.x86_64. >2014-08-10 14:04:58,610 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: attr-2.4.47-3.fc20.x86_64 >2014-08-10 14:04:58,618 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: attr-2.4.47-3.fc20.x86_64. >2014-08-10 14:04:58,619 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: audit-libs-python-2.3.7-1.fc20.x86_64 >2014-08-10 14:04:58,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: audit-libs-python-2.3.7-1.fc20.x86_64. >2014-08-10 14:04:58,627 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: augeas-1.2.0-2.fc20.x86_64 >2014-08-10 14:04:58,635 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: augeas-1.2.0-2.fc20.x86_64. >2014-08-10 14:04:58,635 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: augeas-libs-1.2.0-2.fc20.x86_64 >2014-08-10 14:04:58,643 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: augeas-libs-1.2.0-2.fc20.x86_64. >2014-08-10 14:04:58,644 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: autogen-libopts-5.18-1.fc20.x86_64 >2014-08-10 14:04:58,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: autogen-libopts-5.18-1.fc20.x86_64. >2014-08-10 14:04:58,652 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: binutils-2.23.88.0.1-13.fc20.x86_64 >2014-08-10 14:04:58,660 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: binutils-2.23.88.0.1-13.fc20.x86_64. >2014-08-10 14:04:58,669 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: bluez-libs-5.18-1.fc20.x86_64 >2014-08-10 14:04:58,676 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: bluez-libs-5.18-1.fc20.x86_64. >2014-08-10 14:04:58,677 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: boost-system-1.54.0-9.fc20.x86_64 >2014-08-10 14:04:58,684 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: boost-system-1.54.0-9.fc20.x86_64. >2014-08-10 14:04:58,685 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: boost-thread-1.54.0-9.fc20.x86_64 >2014-08-10 14:04:58,693 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: boost-thread-1.54.0-9.fc20.x86_64. >2014-08-10 14:04:58,693 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: bridge-utils-1.5-8.fc20.x86_64 >2014-08-10 14:04:58,701 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: bridge-utils-1.5-8.fc20.x86_64. >2014-08-10 14:04:58,701 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: brlapi-0.6.0-9.fc20.x86_64 >2014-08-10 14:04:58,709 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: brlapi-0.6.0-9.fc20.x86_64. >2014-08-10 14:04:58,714 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: brltty-4.5-9.fc20.x86_64 >2014-08-10 14:04:58,743 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: brltty-4.5-9.fc20.x86_64. >2014-08-10 14:04:58,743 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: btrfs-progs-3.14.2-3.fc20.x86_64 >2014-08-10 14:04:58,850 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: btrfs-progs-3.14.2-3.fc20.x86_64. >2014-08-10 14:04:58,850 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: bzip2-1.0.6-9.fc20.x86_64 >2014-08-10 14:04:58,858 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: bzip2-1.0.6-9.fc20.x86_64. >2014-08-10 14:04:58,859 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: cairo-1.13.1-0.1.git337ab1f.fc20.x86_64 >2014-08-10 14:04:58,866 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: cairo-1.13.1-0.1.git337ab1f.fc20.x86_64. >2014-08-10 14:04:58,867 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: cairo-gobject-1.13.1-0.1.git337ab1f.fc20.x86_64 >2014-08-10 14:04:58,875 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: cairo-gobject-1.13.1-0.1.git337ab1f.fc20.x86_64. >2014-08-10 14:04:58,875 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: celt051-0.5.1.3-7.fc20.x86_64 >2014-08-10 14:04:58,883 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: celt051-0.5.1.3-7.fc20.x86_64. >2014-08-10 14:04:58,884 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: ceph-libs-0.81.0-5.fc20.x86_64 >2014-08-10 14:04:58,891 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: ceph-libs-0.81.0-5.fc20.x86_64. >2014-08-10 14:04:58,892 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: checkpolicy-2.1.12-5.fc20.x86_64 >2014-08-10 14:04:58,900 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: checkpolicy-2.1.12-5.fc20.x86_64. >2014-08-10 14:04:58,900 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: colord-libs-1.1.7-1.fc20.x86_64 >2014-08-10 14:04:58,908 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: colord-libs-1.1.7-1.fc20.x86_64. >2014-08-10 14:04:58,909 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: corosync-2.3.3-2.fc20.x86_64 >2014-08-10 14:04:58,916 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: corosync-2.3.3-2.fc20.x86_64. >2014-08-10 14:04:58,917 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: corosynclib-2.3.3-2.fc20.x86_64 >2014-08-10 14:04:58,925 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: corosynclib-2.3.3-2.fc20.x86_64. >2014-08-10 14:04:58,925 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: cryptopp-5.6.2-3.fc20.x86_64 >2014-08-10 14:04:58,933 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: cryptopp-5.6.2-3.fc20.x86_64. >2014-08-10 14:04:58,933 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: cryptsetup-1.6.4-1.fc20.x86_64 >2014-08-10 14:04:58,941 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: cryptsetup-1.6.4-1.fc20.x86_64. >2014-08-10 14:04:58,942 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:cups-libs-1.7.4-3.fc20.x86_64 >2014-08-10 14:04:58,949 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:cups-libs-1.7.4-3.fc20.x86_64. >2014-08-10 14:04:58,950 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: cyrus-sasl-2.1.26-14.fc20.x86_64 >2014-08-10 14:04:58,958 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: cyrus-sasl-2.1.26-14.fc20.x86_64. >2014-08-10 14:04:58,958 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: cyrus-sasl-md5-2.1.26-14.fc20.x86_64 >2014-08-10 14:04:58,966 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: cyrus-sasl-md5-2.1.26-14.fc20.x86_64. >2014-08-10 14:04:58,966 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: desktop-file-utils-0.22-1.fc20.x86_64 >2014-08-10 14:04:58,974 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: desktop-file-utils-0.22-1.fc20.x86_64. >2014-08-10 14:04:58,975 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: device-mapper-multipath-0.4.9-56.fc20.x86_64 >2014-08-10 14:04:58,983 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: device-mapper-multipath-0.4.9-56.fc20.x86_64. >2014-08-10 14:04:58,983 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: device-mapper-multipath-libs-0.4.9-56.fc20.x86_64 >2014-08-10 14:04:58,991 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: device-mapper-multipath-libs-0.4.9-56.fc20.x86_64. >2014-08-10 14:04:58,991 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: dosfstools-3.0.26-1.fc20.x86_64 >2014-08-10 14:04:58,999 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: dosfstools-3.0.26-1.fc20.x86_64. >2014-08-10 14:04:59,000 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: ed-1.10-1.fc20.x86_64 >2014-08-10 14:04:59,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: ed-1.10-1.fc20.x86_64. >2014-08-10 14:04:59,008 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:emacs-filesystem-24.3-17.fc20.noarch >2014-08-10 14:04:59,016 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:emacs-filesystem-24.3-17.fc20.noarch. >2014-08-10 14:04:59,016 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 2:ethtool-3.10-2.fc20.x86_64 >2014-08-10 14:04:59,024 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 2:ethtool-3.10-2.fc20.x86_64. >2014-08-10 14:04:59,025 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-all-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,032 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-all-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,033 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-alom-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,041 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-alom-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,042 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-amt-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,049 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-amt-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,049 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-apc-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-apc-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,058 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-apc-snmp-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,065 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-apc-snmp-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,066 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-bladecenter-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,074 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-bladecenter-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,074 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-brocade-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,082 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-brocade-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,083 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-cisco-mds-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,090 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-cisco-mds-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,091 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-cisco-ucs-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,099 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-cisco-ucs-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,099 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-common-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,107 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-common-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,107 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-docker-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,115 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-docker-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,116 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-drac5-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,123 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-drac5-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,124 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-eaton-snmp-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,132 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-eaton-snmp-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,132 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-eps-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,140 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-eps-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,141 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-hpblade-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,148 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-hpblade-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,149 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-ibmblade-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,157 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-ibmblade-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,157 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-ifmib-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,165 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-ifmib-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,165 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-ilo-mp-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,173 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-ilo-mp-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,174 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-ilo2-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,181 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-ilo2-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,182 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-intelmodular-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,190 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-intelmodular-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,190 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-ipdu-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,198 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-ipdu-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,199 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-ipmilan-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,206 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-ipmilan-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,207 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-kdump-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,215 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-kdump-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,215 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-ldom-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,223 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-ldom-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,223 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-lpar-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,231 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-lpar-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,232 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-netio-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,240 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-netio-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,240 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-ovh-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,248 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-ovh-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,248 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-rhevm-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,256 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-rhevm-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,257 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-rsa-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,264 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-rsa-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,265 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-rsb-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,273 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-rsb-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,273 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-scsi-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,281 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-scsi-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,281 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-vmware-soap-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,289 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-vmware-soap-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,290 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fence-agents-wti-4.0.10-1.fc20.x86_64 >2014-08-10 14:04:59,298 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fence-agents-wti-4.0.10-1.fc20.x86_64. >2014-08-10 14:04:59,298 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: flac-libs-1.3.0-3.fc20.x86_64 >2014-08-10 14:04:59,306 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: flac-libs-1.3.0-3.fc20.x86_64. >2014-08-10 14:04:59,306 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fontconfig-2.11.0-2.fc20.x86_64 >2014-08-10 14:04:59,314 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fontconfig-2.11.0-2.fc20.x86_64. >2014-08-10 14:04:59,315 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fontpackages-filesystem-1.44-9.fc20.noarch >2014-08-10 14:04:59,323 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fontpackages-filesystem-1.44-9.fc20.noarch. >2014-08-10 14:04:59,323 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fuse-2.9.3-2.fc20.x86_64 >2014-08-10 14:04:59,331 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fuse-2.9.3-2.fc20.x86_64. >2014-08-10 14:04:59,331 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: fuse-libs-2.9.3-2.fc20.x86_64 >2014-08-10 14:04:59,339 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: fuse-libs-2.9.3-2.fc20.x86_64. >2014-08-10 14:04:59,340 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: gdisk-0.8.10-2.fc20.x86_64 >2014-08-10 14:04:59,347 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: gdisk-0.8.10-2.fc20.x86_64. >2014-08-10 14:04:59,348 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: gdk-pixbuf2-2.30.3-1.fc20.x86_64 >2014-08-10 14:04:59,356 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: gdk-pixbuf2-2.30.3-1.fc20.x86_64. >2014-08-10 14:04:59,356 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: genisoimage-1.1.11-22.fc20.x86_64 >2014-08-10 14:04:59,381 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: genisoimage-1.1.11-22.fc20.x86_64. >2014-08-10 14:04:59,381 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: ghostscript-9.14-3.fc20.x86_64 >2014-08-10 14:04:59,389 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: ghostscript-9.14-3.fc20.x86_64. >2014-08-10 14:04:59,389 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: ghostscript-fonts-5.50-32.fc20.noarch >2014-08-10 14:04:59,397 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: ghostscript-fonts-5.50-32.fc20.noarch. >2014-08-10 14:04:59,398 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: glusterfs-3.5.1-1.fc20.x86_64 >2014-08-10 14:04:59,405 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: glusterfs-3.5.1-1.fc20.x86_64. >2014-08-10 14:04:59,406 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: glusterfs-api-3.5.1-1.fc20.x86_64 >2014-08-10 14:04:59,414 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: glusterfs-api-3.5.1-1.fc20.x86_64. >2014-08-10 14:04:59,414 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: glusterfs-cli-3.5.1-1.fc20.x86_64 >2014-08-10 14:04:59,422 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: glusterfs-cli-3.5.1-1.fc20.x86_64. >2014-08-10 14:04:59,422 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: glusterfs-fuse-3.5.1-1.fc20.x86_64 >2014-08-10 14:04:59,430 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: glusterfs-fuse-3.5.1-1.fc20.x86_64. >2014-08-10 14:04:59,431 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: glusterfs-libs-3.5.1-1.fc20.x86_64 >2014-08-10 14:04:59,438 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: glusterfs-libs-3.5.1-1.fc20.x86_64. >2014-08-10 14:04:59,439 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: glusterfs-rdma-3.5.1-1.fc20.x86_64 >2014-08-10 14:04:59,447 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: glusterfs-rdma-3.5.1-1.fc20.x86_64. >2014-08-10 14:04:59,447 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: gnupg-1.4.18-1.fc20.x86_64 >2014-08-10 14:04:59,455 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: gnupg-1.4.18-1.fc20.x86_64. >2014-08-10 14:04:59,456 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: gnutls-dane-3.1.25-1.fc20.x86_64 >2014-08-10 14:04:59,463 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: gnutls-dane-3.1.25-1.fc20.x86_64. >2014-08-10 14:04:59,464 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: gnutls-utils-3.1.25-1.fc20.x86_64 >2014-08-10 14:04:59,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: gnutls-utils-3.1.25-1.fc20.x86_64. >2014-08-10 14:04:59,472 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: graphite2-1.2.2-4.fc20.x86_64 >2014-08-10 14:04:59,480 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: graphite2-1.2.2-4.fc20.x86_64. >2014-08-10 14:04:59,480 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: gsm-1.0.13-10.fc20.x86_64 >2014-08-10 14:04:59,488 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: gsm-1.0.13-10.fc20.x86_64. >2014-08-10 14:04:59,504 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: gtk2-2.24.22-2.fc20.x86_64 >2014-08-10 14:04:59,513 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: gtk2-2.24.22-2.fc20.x86_64. >2014-08-10 14:04:59,570 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: gtk3-3.10.9-1.fc20.x86_64 >2014-08-10 14:04:59,629 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: gtk3-3.10.9-1.fc20.x86_64. >2014-08-10 14:04:59,630 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: harfbuzz-0.9.27-1.fc20.x86_64 >2014-08-10 14:04:59,637 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: harfbuzz-0.9.27-1.fc20.x86_64. >2014-08-10 14:04:59,638 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: hexedit-1.2.13-4.fc20.x86_64 >2014-08-10 14:04:59,646 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: hexedit-1.2.13-4.fc20.x86_64. >2014-08-10 14:04:59,646 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: hicolor-icon-theme-0.13-1.fc20.noarch >2014-08-10 14:04:59,654 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: hicolor-icon-theme-0.13-1.fc20.noarch. >2014-08-10 14:04:59,655 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: hivex-1.3.8-3.fc20.x86_64 >2014-08-10 14:04:59,662 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: hivex-1.3.8-3.fc20.x86_64. >2014-08-10 14:04:59,663 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: icoutils-0.31.0-3.fc20.x86_64 >2014-08-10 14:04:59,670 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: icoutils-0.31.0-3.fc20.x86_64. >2014-08-10 14:05:00,205 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: ioprocess-0.5.0-1.fc20.x86_64 >2014-08-10 14:05:00,242 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: ioprocess-0.5.0-1.fc20.x86_64. >2014-08-10 14:05:00,246 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: ipmitool-1.8.13-3.fc20.x86_64 >2014-08-10 14:05:00,292 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: ipmitool-1.8.13-3.fc20.x86_64. >2014-08-10 14:05:00,292 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: iptables-services-1.4.19.1-1.fc20.x86_64 >2014-08-10 14:05:00,300 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: iptables-services-1.4.19.1-1.fc20.x86_64. >2014-08-10 14:05:00,300 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: ipxe-roms-qemu-20140303-1.gitff1e7fc7.fc20.noarch >2014-08-10 14:05:00,308 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: ipxe-roms-qemu-20140303-1.gitff1e7fc7.fc20.noarch. >2014-08-10 14:05:00,309 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: jasper-libs-1.900.1-25.fc20.x86_64 >2014-08-10 14:05:00,316 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: jasper-libs-1.900.1-25.fc20.x86_64. >2014-08-10 14:05:00,317 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: jbigkit-libs-2.0-10.fc20.x86_64 >2014-08-10 14:05:00,325 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: jbigkit-libs-2.0-10.fc20.x86_64. >2014-08-10 14:05:00,325 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: json-c-0.11-6.fc20.x86_64 >2014-08-10 14:05:00,333 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: json-c-0.11-6.fc20.x86_64. >2014-08-10 14:05:00,333 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: json-glib-0.16.2-1.fc20.x86_64 >2014-08-10 14:05:00,341 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: json-glib-0.16.2-1.fc20.x86_64. >2014-08-10 14:05:00,342 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: kernel-tools-3.15.8-200.fc20.x86_64 >2014-08-10 14:05:00,349 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: kernel-tools-3.15.8-200.fc20.x86_64. >2014-08-10 14:05:00,350 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: kernel-tools-libs-3.15.8-200.fc20.x86_64 >2014-08-10 14:05:00,358 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: kernel-tools-libs-3.15.8-200.fc20.x86_64. >2014-08-10 14:05:00,358 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: keyutils-1.5.9-1.fc20.x86_64 >2014-08-10 14:05:00,366 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: keyutils-1.5.9-1.fc20.x86_64. >2014-08-10 14:05:00,366 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: lcms2-2.6-1.fc20.x86_64 >2014-08-10 14:05:00,374 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: lcms2-2.6-1.fc20.x86_64. >2014-08-10 14:05:00,375 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: ldns-1.6.16-6.fc20.x86_64 >2014-08-10 14:05:00,383 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: ldns-1.6.16-6.fc20.x86_64. >2014-08-10 14:05:00,383 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: leveldb-1.12.0-5.fc20.x86_64 >2014-08-10 14:05:00,391 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: leveldb-1.12.0-5.fc20.x86_64. >2014-08-10 14:05:00,391 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libICE-1.0.8-6.fc20.x86_64 >2014-08-10 14:05:00,399 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libICE-1.0.8-6.fc20.x86_64. >2014-08-10 14:05:00,400 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libSM-1.2.1-6.fc20.x86_64 >2014-08-10 14:05:00,407 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libSM-1.2.1-6.fc20.x86_64. >2014-08-10 14:05:00,408 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libX11-1.6.1-1.fc20.x86_64 >2014-08-10 14:05:00,416 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libX11-1.6.1-1.fc20.x86_64. >2014-08-10 14:05:00,416 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libX11-common-1.6.1-1.fc20.noarch >2014-08-10 14:05:00,424 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libX11-common-1.6.1-1.fc20.noarch. >2014-08-10 14:05:00,425 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXau-1.0.8-2.fc20.x86_64 >2014-08-10 14:05:00,432 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXau-1.0.8-2.fc20.x86_64. >2014-08-10 14:05:00,433 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXcomposite-0.4.4-4.fc20.x86_64 >2014-08-10 14:05:00,441 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXcomposite-0.4.4-4.fc20.x86_64. >2014-08-10 14:05:00,441 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXcursor-1.1.14-2.fc20.x86_64 >2014-08-10 14:05:00,449 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXcursor-1.1.14-2.fc20.x86_64. >2014-08-10 14:05:00,449 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXdamage-1.1.4-4.fc20.x86_64 >2014-08-10 14:05:00,457 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXdamage-1.1.4-4.fc20.x86_64. >2014-08-10 14:05:00,458 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXevie-1.0.3-7.fc20.x86_64 >2014-08-10 14:05:00,465 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXevie-1.0.3-7.fc20.x86_64. >2014-08-10 14:05:00,466 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXext-1.3.2-2.fc20.x86_64 >2014-08-10 14:05:00,474 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXext-1.3.2-2.fc20.x86_64. >2014-08-10 14:05:00,474 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXfixes-5.0.1-2.fc20.x86_64 >2014-08-10 14:05:00,482 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXfixes-5.0.1-2.fc20.x86_64. >2014-08-10 14:05:00,483 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXfont-1.4.8-1.fc20.x86_64 >2014-08-10 14:05:00,490 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXfont-1.4.8-1.fc20.x86_64. >2014-08-10 14:05:00,491 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXft-2.3.1-5.fc20.x86_64 >2014-08-10 14:05:00,499 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXft-2.3.1-5.fc20.x86_64. >2014-08-10 14:05:00,499 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXi-1.7.4-1.fc20.x86_64 >2014-08-10 14:05:00,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXi-1.7.4-1.fc20.x86_64. >2014-08-10 14:05:00,507 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXinerama-1.1.3-2.fc20.x86_64 >2014-08-10 14:05:00,515 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXinerama-1.1.3-2.fc20.x86_64. >2014-08-10 14:05:00,516 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXrandr-1.4.1-2.fc20.x86_64 >2014-08-10 14:05:00,524 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXrandr-1.4.1-2.fc20.x86_64. >2014-08-10 14:05:00,524 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXrender-0.9.8-2.fc20.x86_64 >2014-08-10 14:05:00,548 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXrender-0.9.8-2.fc20.x86_64. >2014-08-10 14:05:00,549 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXt-1.1.4-7.fc20.x86_64 >2014-08-10 14:05:00,557 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXt-1.1.4-7.fc20.x86_64. >2014-08-10 14:05:00,557 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXtst-1.2.2-2.fc20.x86_64 >2014-08-10 14:05:00,565 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXtst-1.2.2-2.fc20.x86_64. >2014-08-10 14:05:00,565 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libXxf86vm-1.1.3-2.fc20.x86_64 >2014-08-10 14:05:00,573 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libXxf86vm-1.1.3-2.fc20.x86_64. >2014-08-10 14:05:00,574 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libaio-0.3.109-8.fc20.x86_64 >2014-08-10 14:05:00,606 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libaio-0.3.109-8.fc20.x86_64. >2014-08-10 14:05:00,607 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libasyncns-0.8-6.fc20.x86_64 >2014-08-10 14:05:00,615 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libasyncns-0.8-6.fc20.x86_64. >2014-08-10 14:05:00,615 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libcgroup-0.38-8.fc20.x86_64 >2014-08-10 14:05:00,623 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libcgroup-0.38-8.fc20.x86_64. >2014-08-10 14:05:00,624 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libconfig-1.4.9-5.fc20.x86_64 >2014-08-10 14:05:00,631 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libconfig-1.4.9-5.fc20.x86_64. >2014-08-10 14:05:00,632 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libevent-2.0.21-3.fc20.x86_64 >2014-08-10 14:05:00,640 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libevent-2.0.21-3.fc20.x86_64. >2014-08-10 14:05:00,640 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libfdt-1.4.0-2.fc20.x86_64 >2014-08-10 14:05:00,648 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libfdt-1.4.0-2.fc20.x86_64. >2014-08-10 14:05:00,648 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libfontenc-1.1.1-4.fc20.x86_64 >2014-08-10 14:05:00,656 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libfontenc-1.1.1-4.fc20.x86_64. >2014-08-10 14:05:00,657 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:libguestfs-1.26.7-2.fc20.x86_64 >2014-08-10 14:05:00,664 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:libguestfs-1.26.7-2.fc20.x86_64. >2014-08-10 14:05:00,665 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:libguestfs-tools-c-1.26.7-2.fc20.x86_64 >2014-08-10 14:05:00,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:libguestfs-tools-c-1.26.7-2.fc20.x86_64. >2014-08-10 14:05:00,673 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libgusb-0.1.6-2.fc20.x86_64 >2014-08-10 14:05:00,681 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libgusb-0.1.6-2.fc20.x86_64. >2014-08-10 14:05:00,681 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libibverbs-1.1.7-3.fc20.x86_64 >2014-08-10 14:05:00,689 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libibverbs-1.1.7-3.fc20.x86_64. >2014-08-10 14:05:00,690 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libicu-50.1.2-10.fc20.x86_64 >2014-08-10 14:05:00,698 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libicu-50.1.2-10.fc20.x86_64. >2014-08-10 14:05:00,721 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libiscsi-1.9.0-4.fc20.x86_64 >2014-08-10 14:05:00,731 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libiscsi-1.9.0-4.fc20.x86_64. >2014-08-10 14:05:00,731 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libjpeg-turbo-1.3.1-2.fc20.x86_64 >2014-08-10 14:05:00,739 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libjpeg-turbo-1.3.1-2.fc20.x86_64. >2014-08-10 14:05:00,740 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libldm-0.2.3-4.fc20.x86_64 >2014-08-10 14:05:00,747 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libldm-0.2.3-4.fc20.x86_64. >2014-08-10 14:05:00,748 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libnfsidmap-0.25-8.fc20.x86_64 >2014-08-10 14:05:00,756 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libnfsidmap-0.25-8.fc20.x86_64. >2014-08-10 14:05:00,756 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libnl-1.1.4-3.fc20.x86_64 >2014-08-10 14:05:00,764 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libnl-1.1.4-3.fc20.x86_64. >2014-08-10 14:05:00,764 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 2:libogg-1.3.0-6.fc20.x86_64 >2014-08-10 14:05:00,772 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 2:libogg-1.3.0-6.fc20.x86_64. >2014-08-10 14:05:00,773 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libosinfo-0.2.9-1.fc20.x86_64 >2014-08-10 14:05:00,781 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libosinfo-0.2.9-1.fc20.x86_64. >2014-08-10 14:05:00,781 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libqb-0.17.0-1.fc20.x86_64 >2014-08-10 14:05:00,789 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libqb-0.17.0-1.fc20.x86_64. >2014-08-10 14:05:00,789 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: librdmacm-1.0.17-2.fc20.x86_64 >2014-08-10 14:05:00,797 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: librdmacm-1.0.17-2.fc20.x86_64. >2014-08-10 14:05:00,798 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libreport-filesystem-2.2.3-1.fc20.x86_64 >2014-08-10 14:05:00,805 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libreport-filesystem-2.2.3-1.fc20.x86_64. >2014-08-10 14:05:00,806 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libseccomp-2.1.0-1.fc20.x86_64 >2014-08-10 14:05:00,814 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libseccomp-2.1.0-1.fc20.x86_64. >2014-08-10 14:05:00,814 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libsemanage-python-2.1.10-14.fc20.x86_64 >2014-08-10 14:05:00,822 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libsemanage-python-2.1.10-14.fc20.x86_64. >2014-08-10 14:05:00,822 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libsndfile-1.0.25-8.fc20.x86_64 >2014-08-10 14:05:00,830 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libsndfile-1.0.25-8.fc20.x86_64. >2014-08-10 14:05:00,831 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libthai-0.1.20-1.fc20.x86_64 >2014-08-10 14:05:00,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libthai-0.1.20-1.fc20.x86_64. >2014-08-10 14:05:00,839 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libtiff-4.0.3-15.fc20.x86_64 >2014-08-10 14:05:00,847 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libtiff-4.0.3-15.fc20.x86_64. >2014-08-10 14:05:00,847 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libtirpc-0.2.4-3.0.fc20.x86_64 >2014-08-10 14:05:00,855 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libtirpc-0.2.4-3.0.fc20.x86_64. >2014-08-10 14:05:00,856 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libusal-1.1.11-22.fc20.x86_64 >2014-08-10 14:05:00,863 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libusal-1.1.11-22.fc20.x86_64. >2014-08-10 14:05:00,864 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:libusb-0.1.5-3.fc20.x86_64 >2014-08-10 14:05:00,872 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:libusb-0.1.5-3.fc20.x86_64. >2014-08-10 14:05:00,872 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libusbx-1.0.19-1.fc20.x86_64 >2014-08-10 14:05:00,880 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libusbx-1.0.19-1.fc20.x86_64. >2014-08-10 14:05:00,900 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-client-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:00,913 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-client-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:00,945 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-daemon-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:00,954 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-daemon-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:00,956 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-daemon-config-nwfilter-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:00,963 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-daemon-config-nwfilter-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:00,965 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-daemon-driver-interface-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:00,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-daemon-driver-interface-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:00,972 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-daemon-driver-network-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:00,979 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-daemon-driver-network-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:00,980 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-daemon-driver-nodedev-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:00,988 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-daemon-driver-nodedev-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:00,988 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-daemon-driver-nwfilter-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:00,996 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-daemon-driver-nwfilter-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:00,996 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-daemon-driver-qemu-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:01,004 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-daemon-driver-qemu-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:01,005 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-daemon-driver-secret-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:01,012 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-daemon-driver-secret-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:01,013 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-daemon-driver-storage-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:01,021 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-daemon-driver-storage-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:01,021 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-daemon-kvm-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:01,029 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-daemon-kvm-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:01,030 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-lock-sanlock-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:01,037 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-lock-sanlock-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:01,038 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libvirt-python-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:05:01,046 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libvirt-python-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:05:01,046 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:libvorbis-1.3.4-1.fc20.x86_64 >2014-08-10 14:05:01,054 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:libvorbis-1.3.4-1.fc20.x86_64. >2014-08-10 14:05:01,054 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libwayland-client-1.2.0-3.fc20.x86_64 >2014-08-10 14:05:01,062 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libwayland-client-1.2.0-3.fc20.x86_64. >2014-08-10 14:05:01,063 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libwayland-cursor-1.2.0-3.fc20.x86_64 >2014-08-10 14:05:01,070 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libwayland-cursor-1.2.0-3.fc20.x86_64. >2014-08-10 14:05:01,071 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libwayland-server-1.2.0-3.fc20.x86_64 >2014-08-10 14:05:01,079 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libwayland-server-1.2.0-3.fc20.x86_64. >2014-08-10 14:05:01,079 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libwsman1-2.4.3-1.fc20.x86_64 >2014-08-10 14:05:01,087 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libwsman1-2.4.3-1.fc20.x86_64. >2014-08-10 14:05:01,088 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libxcb-1.9.1-3.fc20.x86_64 >2014-08-10 14:05:01,095 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libxcb-1.9.1-3.fc20.x86_64. >2014-08-10 14:05:01,096 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libxkbcommon-0.3.1-1.fc20.x86_64 >2014-08-10 14:05:01,104 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libxkbcommon-0.3.1-1.fc20.x86_64. >2014-08-10 14:05:01,104 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libxml2-python-2.9.1-2.fc20.x86_64 >2014-08-10 14:05:01,112 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libxml2-python-2.9.1-2.fc20.x86_64. >2014-08-10 14:05:01,113 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: libxslt-1.1.28-5.fc20.x86_64 >2014-08-10 14:05:01,128 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: libxslt-1.1.28-5.fc20.x86_64. >2014-08-10 14:05:01,129 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: logrotate-3.8.7-1.fc20.x86_64 >2014-08-10 14:05:01,137 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: logrotate-3.8.7-1.fc20.x86_64. >2014-08-10 14:05:01,137 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: lsscsi-0.27-2.fc20.x86_64 >2014-08-10 14:05:01,145 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: lsscsi-0.27-2.fc20.x86_64. >2014-08-10 14:05:01,145 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: lzo-2.08-1.fc20.x86_64 >2014-08-10 14:05:01,153 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: lzo-2.08-1.fc20.x86_64. >2014-08-10 14:05:01,154 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: lzop-1.03-9.fc20.x86_64 >2014-08-10 14:05:01,162 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: lzop-1.03-9.fc20.x86_64. >2014-08-10 14:05:01,162 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: m2crypto-0.21.1-13.fc20.x86_64 >2014-08-10 14:05:01,170 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: m2crypto-0.21.1-13.fc20.x86_64. >2014-08-10 14:05:01,170 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: mailcap-2.1.42-1.fc20.noarch >2014-08-10 14:05:01,178 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: mailcap-2.1.42-1.fc20.noarch. >2014-08-10 14:05:01,179 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: mdadm-3.3-4.fc20.x86_64 >2014-08-10 14:05:01,186 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: mdadm-3.3-4.fc20.x86_64. >2014-08-10 14:05:01,187 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: mesa-libEGL-10.1.5-1.20140607.fc20.x86_64 >2014-08-10 14:05:01,195 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: mesa-libEGL-10.1.5-1.20140607.fc20.x86_64. >2014-08-10 14:05:01,195 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: mesa-libGL-10.1.5-1.20140607.fc20.x86_64 >2014-08-10 14:05:01,203 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: mesa-libGL-10.1.5-1.20140607.fc20.x86_64. >2014-08-10 14:05:01,204 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: mesa-libgbm-10.1.5-1.20140607.fc20.x86_64 >2014-08-10 14:05:01,211 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: mesa-libgbm-10.1.5-1.20140607.fc20.x86_64. >2014-08-10 14:05:01,212 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: mesa-libglapi-10.1.5-1.20140607.fc20.x86_64 >2014-08-10 14:05:01,220 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: mesa-libglapi-10.1.5-1.20140607.fc20.x86_64. >2014-08-10 14:05:01,220 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: mom-0.4.1-1.fc20.noarch >2014-08-10 14:05:01,228 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: mom-0.4.1-1.fc20.noarch. >2014-08-10 14:05:01,228 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: mtools-4.0.18-4.fc20.x86_64 >2014-08-10 14:05:01,236 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: mtools-4.0.18-4.fc20.x86_64. >2014-08-10 14:05:01,237 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:net-snmp-libs-5.7.2-17.fc20.x86_64 >2014-08-10 14:05:01,245 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:net-snmp-libs-5.7.2-17.fc20.x86_64. >2014-08-10 14:05:01,245 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:net-snmp-utils-5.7.2-17.fc20.x86_64 >2014-08-10 14:05:01,253 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:net-snmp-utils-5.7.2-17.fc20.x86_64. >2014-08-10 14:05:01,253 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: netcf-libs-0.2.3-6.fc20.x86_64 >2014-08-10 14:05:01,261 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: netcf-libs-0.2.3-6.fc20.x86_64. >2014-08-10 14:05:01,262 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: netpbm-10.61.02-8.fc20.x86_64 >2014-08-10 14:05:01,269 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: netpbm-10.61.02-8.fc20.x86_64. >2014-08-10 14:05:01,281 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: netpbm-progs-10.61.02-8.fc20.x86_64 >2014-08-10 14:05:01,294 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: netpbm-progs-10.61.02-8.fc20.x86_64. >2014-08-10 14:05:01,298 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:nfs-utils-1.3.0-2.1.fc20.x86_64 >2014-08-10 14:05:01,311 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:nfs-utils-1.3.0-2.1.fc20.x86_64. >2014-08-10 14:05:01,311 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 2:nmap-ncat-6.45-1.fc20.x86_64 >2014-08-10 14:05:01,319 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 2:nmap-ncat-6.45-1.fc20.x86_64. >2014-08-10 14:05:01,320 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 2:ntfs-3g-2014.2.15-1.fc20.x86_64 >2014-08-10 14:05:01,327 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 2:ntfs-3g-2014.2.15-1.fc20.x86_64. >2014-08-10 14:05:01,328 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 2:ntfsprogs-2014.2.15-1.fc20.x86_64 >2014-08-10 14:05:01,336 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 2:ntfsprogs-2014.2.15-1.fc20.x86_64. >2014-08-10 14:05:01,343 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: ntp-4.2.6p5-18.fc20.x86_64 >2014-08-10 14:05:01,352 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: ntp-4.2.6p5-18.fc20.x86_64. >2014-08-10 14:05:01,353 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: ntpdate-4.2.6p5-18.fc20.x86_64 >2014-08-10 14:05:01,361 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: ntpdate-4.2.6p5-18.fc20.x86_64. >2014-08-10 14:05:01,372 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: numactl-2.0.9-1.fc20.x86_64 >2014-08-10 14:05:01,385 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: numactl-2.0.9-1.fc20.x86_64. >2014-08-10 14:05:01,386 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: numactl-libs-2.0.9-1.fc20.x86_64 >2014-08-10 14:05:01,394 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: numactl-libs-2.0.9-1.fc20.x86_64. >2014-08-10 14:05:01,394 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: numad-0.5-12.20130814git.fc20.x86_64 >2014-08-10 14:05:01,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: numad-0.5-12.20130814git.fc20.x86_64. >2014-08-10 14:05:01,403 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: opus-1.1-1.fc20.x86_64 >2014-08-10 14:05:01,410 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: opus-1.1-1.fc20.x86_64. >2014-08-10 14:05:01,415 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: pango-1.36.1-3.fc20.x86_64 >2014-08-10 14:05:01,427 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: pango-1.36.1-3.fc20.x86_64. >2014-08-10 14:05:01,427 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Business-ISBN-2.06-4.fc20.noarch >2014-08-10 14:05:01,435 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Business-ISBN-2.06-4.fc20.noarch. >2014-08-10 14:05:01,436 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Business-ISBN-Data-20120719.001-4.fc20.noarch >2014-08-10 14:05:01,443 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Business-ISBN-Data-20120719.001-4.fc20.noarch. >2014-08-10 14:05:01,444 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Class-Inspector-1.28-5.fc20.noarch >2014-08-10 14:05:01,452 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Class-Inspector-1.28-5.fc20.noarch. >2014-08-10 14:05:01,638 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Compress-Raw-Bzip2-2.062-2.fc20.x86_64 >2014-08-10 14:05:01,661 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Compress-Raw-Bzip2-2.062-2.fc20.x86_64. >2014-08-10 14:05:01,662 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Compress-Raw-Zlib-2.062-2.fc20.x86_64 >2014-08-10 14:05:01,684 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Compress-Raw-Zlib-2.062-2.fc20.x86_64. >2014-08-10 14:05:01,685 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Convert-BinHex-1.122-1.fc20.noarch >2014-08-10 14:05:01,692 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Convert-BinHex-1.122-1.fc20.noarch. >2014-08-10 14:05:01,693 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Data-Dumper-2.151-1.fc20.x86_64 >2014-08-10 14:05:01,700 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Data-Dumper-2.151-1.fc20.x86_64. >2014-08-10 14:05:01,701 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Digest-1.17-291.fc20.noarch >2014-08-10 14:05:01,708 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Digest-1.17-291.fc20.noarch. >2014-08-10 14:05:01,709 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Digest-MD5-2.53-3.fc20.x86_64 >2014-08-10 14:05:01,717 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Digest-MD5-2.53-3.fc20.x86_64. >2014-08-10 14:05:01,718 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Email-Date-Format-1.002-17.fc20.noarch >2014-08-10 14:05:01,725 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Email-Date-Format-1.002-17.fc20.noarch. >2014-08-10 14:05:01,726 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Encode-Locale-1.03-7.fc20.noarch >2014-08-10 14:05:01,733 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Encode-Locale-1.03-7.fc20.noarch. >2014-08-10 14:05:01,734 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-File-Listing-6.04-7.fc20.noarch >2014-08-10 14:05:01,742 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-File-Listing-6.04-7.fc20.noarch. >2014-08-10 14:05:01,743 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-HTML-Parser-3.71-4.fc20.x86_64 >2014-08-10 14:05:01,750 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-HTML-Parser-3.71-4.fc20.x86_64. >2014-08-10 14:05:01,750 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-HTML-Tagset-3.20-18.fc20.noarch >2014-08-10 14:05:01,758 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-HTML-Tagset-3.20-18.fc20.noarch. >2014-08-10 14:05:01,759 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-HTTP-Cookies-6.01-7.fc20.noarch >2014-08-10 14:05:01,766 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-HTTP-Cookies-6.01-7.fc20.noarch. >2014-08-10 14:05:01,767 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-HTTP-Daemon-6.01-7.fc20.noarch >2014-08-10 14:05:01,775 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-HTTP-Daemon-6.01-7.fc20.noarch. >2014-08-10 14:05:01,775 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-HTTP-Date-6.02-8.fc20.noarch >2014-08-10 14:05:01,783 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-HTTP-Date-6.02-8.fc20.noarch. >2014-08-10 14:05:01,783 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-HTTP-Message-6.06-7.fc20.noarch >2014-08-10 14:05:01,791 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-HTTP-Message-6.06-7.fc20.noarch. >2014-08-10 14:05:01,792 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-HTTP-Negotiate-6.01-7.fc20.noarch >2014-08-10 14:05:01,800 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-HTTP-Negotiate-6.01-7.fc20.noarch. >2014-08-10 14:05:01,800 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-IO-Compress-2.062-2.fc20.noarch >2014-08-10 14:05:01,808 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-IO-Compress-2.062-2.fc20.noarch. >2014-08-10 14:05:01,809 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-IO-HTML-1.001-1.fc20.noarch >2014-08-10 14:05:01,816 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-IO-HTML-1.001-1.fc20.noarch. >2014-08-10 14:05:01,817 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-IO-Socket-IP-0.30-2.fc20.noarch >2014-08-10 14:05:01,825 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-IO-Socket-IP-0.30-2.fc20.noarch. >2014-08-10 14:05:01,825 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-IO-Socket-SSL-1.955-2.fc20.noarch >2014-08-10 14:05:01,833 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-IO-Socket-SSL-1.955-2.fc20.noarch. >2014-08-10 14:05:01,833 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-LWP-MediaTypes-6.02-4.fc20.noarch >2014-08-10 14:05:01,841 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-LWP-MediaTypes-6.02-4.fc20.noarch. >2014-08-10 14:05:01,841 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-MIME-Lite-3.030-1.fc20.noarch >2014-08-10 14:05:01,849 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-MIME-Lite-3.030-1.fc20.noarch. >2014-08-10 14:05:01,850 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-MIME-Types-2.04-1.fc20.noarch >2014-08-10 14:05:01,858 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-MIME-Types-2.04-1.fc20.noarch. >2014-08-10 14:05:01,858 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-MIME-tools-5.505-1.fc20.noarch >2014-08-10 14:05:01,866 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-MIME-tools-5.505-1.fc20.noarch. >2014-08-10 14:05:01,866 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-MailTools-2.12-4.fc20.noarch >2014-08-10 14:05:01,874 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-MailTools-2.12-4.fc20.noarch. >2014-08-10 14:05:01,875 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Net-HTTP-6.06-4.fc20.noarch >2014-08-10 14:05:01,882 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Net-HTTP-6.06-4.fc20.noarch. >2014-08-10 14:05:01,883 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Net-LibIDN-0.12-16.fc20.x86_64 >2014-08-10 14:05:01,891 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Net-LibIDN-0.12-16.fc20.x86_64. >2014-08-10 14:05:01,891 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Net-SMTP-SSL-1.01-16.fc20.noarch >2014-08-10 14:05:01,899 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Net-SMTP-SSL-1.01-16.fc20.noarch. >2014-08-10 14:05:01,899 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-Net-SSLeay-1.55-4.fc20.x86_64 >2014-08-10 14:05:01,907 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-Net-SSLeay-1.55-4.fc20.x86_64. >2014-08-10 14:05:01,908 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-SOAP-Lite-0.716-3.fc20.noarch >2014-08-10 14:05:01,916 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-SOAP-Lite-0.716-3.fc20.noarch. >2014-08-10 14:05:01,916 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-SOAP-Transport-TCP-0.715-11.fc20.noarch >2014-08-10 14:05:01,924 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-SOAP-Transport-TCP-0.715-11.fc20.noarch. >2014-08-10 14:05:01,924 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:perl-TimeDate-2.30-3.fc20.noarch >2014-08-10 14:05:01,932 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:perl-TimeDate-2.30-3.fc20.noarch. >2014-08-10 14:05:01,933 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-URI-1.60-11.fc20.noarch >2014-08-10 14:05:01,940 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-URI-1.60-11.fc20.noarch. >2014-08-10 14:05:01,941 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-WWW-RobotRules-6.02-8.fc20.noarch >2014-08-10 14:05:01,949 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-WWW-RobotRules-6.02-8.fc20.noarch. >2014-08-10 14:05:01,949 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-XML-Parser-2.41-11.fc20.x86_64 >2014-08-10 14:05:01,957 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-XML-Parser-2.41-11.fc20.x86_64. >2014-08-10 14:05:01,958 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-hivex-1.3.8-3.fc20.x86_64 >2014-08-10 14:05:01,965 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-hivex-1.3.8-3.fc20.x86_64. >2014-08-10 14:05:01,966 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: perl-libwww-perl-6.05-3.fc20.noarch >2014-08-10 14:05:01,974 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: perl-libwww-perl-6.05-3.fc20.noarch. >2014-08-10 14:05:01,974 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: pixman-0.30.0-4.fc20.x86_64 >2014-08-10 14:05:01,982 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: pixman-0.30.0-4.fc20.x86_64. >2014-08-10 14:05:01,982 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: pm-utils-1.4.1-26.fc20.x86_64 >2014-08-10 14:05:01,990 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: pm-utils-1.4.1-26.fc20.x86_64. >2014-08-10 14:05:01,991 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: policycoreutils-python-2.2.5-4.fc20.x86_64 >2014-08-10 14:05:01,998 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: policycoreutils-python-2.2.5-4.fc20.x86_64. >2014-08-10 14:05:01,999 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: poppler-data-0.4.6-4.fc20.noarch >2014-08-10 14:05:02,007 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: poppler-data-0.4.6-4.fc20.noarch. >2014-08-10 14:05:02,007 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: psmisc-22.20-3.fc20.x86_64 >2014-08-10 14:05:02,015 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: psmisc-22.20-3.fc20.x86_64. >2014-08-10 14:05:02,015 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: pulseaudio-libs-5.0-7.fc20.x86_64 >2014-08-10 14:05:02,023 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: pulseaudio-libs-5.0-7.fc20.x86_64. >2014-08-10 14:05:02,024 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:pyparted-3.9-4.fc20.x86_64 >2014-08-10 14:05:02,032 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:pyparted-3.9-4.fc20.x86_64. >2014-08-10 14:05:02,032 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-IPy-0.75-6.fc20.noarch >2014-08-10 14:05:02,040 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-IPy-0.75-6.fc20.noarch. >2014-08-10 14:05:02,041 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-backports-1.0-3.fc20.x86_64 >2014-08-10 14:05:02,048 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-backports-1.0-3.fc20.x86_64. >2014-08-10 14:05:02,049 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-backports-ssl_match_hostname-3.4.0.2-1.fc20.noarch >2014-08-10 14:05:02,056 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-backports-ssl_match_hostname-3.4.0.2-1.fc20.noarch. >2014-08-10 14:05:02,057 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-chardet-2.0.1-7.fc20.noarch >2014-08-10 14:05:02,065 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-chardet-2.0.1-7.fc20.noarch. >2014-08-10 14:05:02,065 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-configobj-4.7.2-7.fc20.noarch >2014-08-10 14:05:02,073 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-configobj-4.7.2-7.fc20.noarch. >2014-08-10 14:05:02,073 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-cpopen-1.3-4.fc20.x86_64 >2014-08-10 14:05:02,081 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-cpopen-1.3-4.fc20.x86_64. >2014-08-10 14:05:02,082 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-dmidecode-3.10.13-10.fc20.x86_64 >2014-08-10 14:05:02,090 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-dmidecode-3.10.13-10.fc20.x86_64. >2014-08-10 14:05:02,090 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-ethtool-0.10-1.fc20.x86_64 >2014-08-10 14:05:02,098 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-ethtool-0.10-1.fc20.x86_64. >2014-08-10 14:05:02,098 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-inotify-0.9.4-4.fc20.noarch >2014-08-10 14:05:02,106 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-inotify-0.9.4-4.fc20.noarch. >2014-08-10 14:05:02,794 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-ioprocess-0.5.0-1.fc20.noarch >2014-08-10 14:05:02,802 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-ioprocess-0.5.0-1.fc20.noarch. >2014-08-10 14:05:02,806 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-kitchen-1.1.1-5.fc20.noarch >2014-08-10 14:05:02,818 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-kitchen-1.1.1-5.fc20.noarch. >2014-08-10 14:05:02,836 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-netaddr-0.7.11-1.fc20.noarch >2014-08-10 14:05:02,843 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-netaddr-0.7.11-1.fc20.noarch. >2014-08-10 14:05:02,852 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-pexpect-3.1-1.fc20.noarch >2014-08-10 14:05:02,860 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-pexpect-3.1-1.fc20.noarch. >2014-08-10 14:05:02,860 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-pthreading-0.1.3-3.fc20.noarch >2014-08-10 14:05:02,868 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-pthreading-0.1.3-3.fc20.noarch. >2014-08-10 14:05:02,868 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-pyudev-0.15-5.fc20.noarch >2014-08-10 14:05:02,876 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-pyudev-0.15-5.fc20.noarch. >2014-08-10 14:05:02,877 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-setuptools-1.4.2-1.fc20.noarch >2014-08-10 14:05:02,885 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-setuptools-1.4.2-1.fc20.noarch. >2014-08-10 14:05:02,888 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: python-suds-0.4.1-5.fc20.noarch >2014-08-10 14:05:02,901 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: python-suds-0.4.1-5.fc20.noarch. >2014-08-10 14:05:02,902 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 2:qemu-common-1.6.2-7.fc20.x86_64 >2014-08-10 14:05:02,909 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 2:qemu-common-1.6.2-7.fc20.x86_64. >2014-08-10 14:05:02,912 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 2:qemu-img-1.6.2-7.fc20.x86_64 >2014-08-10 14:05:02,926 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 2:qemu-img-1.6.2-7.fc20.x86_64. >2014-08-10 14:05:02,927 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 2:qemu-kvm-1.6.2-7.fc20.x86_64 >2014-08-10 14:05:02,934 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 2:qemu-kvm-1.6.2-7.fc20.x86_64. >2014-08-10 14:05:02,935 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 2:qemu-kvm-tools-1.6.2-7.fc20.x86_64 >2014-08-10 14:05:02,943 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 2:qemu-kvm-tools-1.6.2-7.fc20.x86_64. >2014-08-10 14:05:02,972 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 2:qemu-system-x86-1.6.2-7.fc20.x86_64 >2014-08-10 14:05:02,984 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 2:qemu-system-x86-1.6.2-7.fc20.x86_64. >2014-08-10 14:05:02,996 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:quota-4.01-11.fc20.x86_64 >2014-08-10 14:05:03,009 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:quota-4.01-11.fc20.x86_64. >2014-08-10 14:05:03,009 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:quota-nls-4.01-11.fc20.noarch >2014-08-10 14:05:03,017 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:quota-nls-4.01-11.fc20.noarch. >2014-08-10 14:05:03,018 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: radvd-1.9.9-1.fc20.x86_64 >2014-08-10 14:05:03,025 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: radvd-1.9.9-1.fc20.x86_64. >2014-08-10 14:05:03,026 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: rpcbind-0.2.1-0.2.fc20.x86_64 >2014-08-10 14:05:03,034 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: rpcbind-0.2.1-0.2.fc20.x86_64. >2014-08-10 14:05:03,034 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: sanlock-3.0.0-2.fc20.x86_64 >2014-08-10 14:05:03,042 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: sanlock-3.0.0-2.fc20.x86_64. >2014-08-10 14:05:03,043 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: sanlock-lib-3.0.0-2.fc20.x86_64 >2014-08-10 14:05:03,050 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: sanlock-lib-3.0.0-2.fc20.x86_64. >2014-08-10 14:05:03,051 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: sanlock-python-3.0.0-2.fc20.x86_64 >2014-08-10 14:05:03,059 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: sanlock-python-3.0.0-2.fc20.x86_64. >2014-08-10 14:05:03,059 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: scrub-2.5.2-4.fc20.x86_64 >2014-08-10 14:05:03,067 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: scrub-2.5.2-4.fc20.x86_64. >2014-08-10 14:05:03,067 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: seabios-bin-1.7.3.1-2.fc20.noarch >2014-08-10 14:05:03,075 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: seabios-bin-1.7.3.1-2.fc20.noarch. >2014-08-10 14:05:03,075 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: seavgabios-bin-1.7.3.1-2.fc20.noarch >2014-08-10 14:05:03,083 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: seavgabios-bin-1.7.3.1-2.fc20.noarch. >2014-08-10 14:05:03,084 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: setools-libs-3.3.7-41.fc20.x86_64 >2014-08-10 14:05:03,092 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: setools-libs-3.3.7-41.fc20.x86_64. >2014-08-10 14:05:03,092 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: sg3_utils-1.37-2.fc20.x86_64 >2014-08-10 14:05:03,100 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: sg3_utils-1.37-2.fc20.x86_64. >2014-08-10 14:05:03,100 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: sg3_utils-libs-1.37-2.fc20.x86_64 >2014-08-10 14:05:03,108 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: sg3_utils-libs-1.37-2.fc20.x86_64. >2014-08-10 14:05:03,109 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:sgabios-bin-0.20110622svn-6.fc20.noarch >2014-08-10 14:05:03,116 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:sgabios-bin-0.20110622svn-6.fc20.noarch. >2014-08-10 14:05:03,117 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: sheepdog-0.3.0-5.fc20.x86_64 >2014-08-10 14:05:03,125 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: sheepdog-0.3.0-5.fc20.x86_64. >2014-08-10 14:05:03,125 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: snappy-1.1.0-2.fc20.x86_64 >2014-08-10 14:05:03,133 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: snappy-1.1.0-2.fc20.x86_64. >2014-08-10 14:05:04,507 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: sos-3.1-1.1.fc20.ovirt.noarch >2014-08-10 14:05:04,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: sos-3.1-1.1.fc20.ovirt.noarch. >2014-08-10 14:05:04,558 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: spice-server-0.12.5-2.fc20.x86_64 >2014-08-10 14:05:04,566 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: spice-server-0.12.5-2.fc20.x86_64. >2014-08-10 14:05:04,566 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: supermin-5.1.9-1.fc20.x86_64 >2014-08-10 14:05:04,574 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: supermin-5.1.9-1.fc20.x86_64. >2014-08-10 14:05:04,575 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: syslinux-4.05-7.fc20.x86_64 >2014-08-10 14:05:04,583 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: syslinux-4.05-7.fc20.x86_64. >2014-08-10 14:05:04,587 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: syslinux-extlinux-4.05-7.fc20.x86_64 >2014-08-10 14:05:04,599 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: syslinux-extlinux-4.05-7.fc20.x86_64. >2014-08-10 14:05:04,600 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: tcp_wrappers-7.6-76.fc20.x86_64 >2014-08-10 14:05:04,607 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: tcp_wrappers-7.6-76.fc20.x86_64. >2014-08-10 14:05:04,608 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:telnet-0.17-58.fc20.x86_64 >2014-08-10 14:05:04,616 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:telnet-0.17-58.fc20.x86_64. >2014-08-10 14:05:04,616 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: tree-1.6.0-11.fc20.x86_64 >2014-08-10 14:05:04,624 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: tree-1.6.0-11.fc20.x86_64. >2014-08-10 14:05:04,625 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: tuned-2.3.0-3.fc20.noarch >2014-08-10 14:05:04,632 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: tuned-2.3.0-3.fc20.noarch. >2014-08-10 14:05:04,633 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: uml_utilities-20070815-12.fc20.x86_64 >2014-08-10 14:05:04,641 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: uml_utilities-20070815-12.fc20.x86_64. >2014-08-10 14:05:04,641 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: unbound-libs-1.4.21-3.fc20.x86_64 >2014-08-10 14:05:04,649 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: unbound-libs-1.4.21-3.fc20.x86_64. >2014-08-10 14:05:04,679 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: urw-fonts-2.4-18.fc20.noarch >2014-08-10 14:05:04,690 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: urw-fonts-2.4-18.fc20.noarch. >2014-08-10 14:05:04,697 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: usbredir-0.6-5.fc20.x86_64 >2014-08-10 14:05:04,707 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: usbredir-0.6-5.fc20.x86_64. >2014-08-10 14:05:05,941 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: vdsm-4.16.1-6.gita4a4614.fc20.x86_64 >2014-08-10 14:05:05,957 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-4.16.1-6.gita4a4614.fc20.x86_64. >2014-08-10 14:05:06,176 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: vdsm-cli-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:05:06,189 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-cli-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:05:06,737 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: vdsm-jsonrpc-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:05:06,777 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-jsonrpc-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:05:07,015 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: vdsm-python-4.16.1-6.gita4a4614.fc20.x86_64 >2014-08-10 14:05:07,025 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-python-4.16.1-6.gita4a4614.fc20.x86_64. >2014-08-10 14:05:07,244 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: vdsm-python-zombiereaper-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:05:07,257 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-python-zombiereaper-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:05:07,466 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: vdsm-xmlrpc-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:05:07,473 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-xmlrpc-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:05:07,652 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: vdsm-yajsonrpc-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:05:07,663 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-yajsonrpc-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:05:07,664 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: virt-what-1.13-2.fc20.x86_64 >2014-08-10 14:05:07,671 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: virt-what-1.13-2.fc20.x86_64. >2014-08-10 14:05:07,674 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: vte-0.28.2-13.fc20.x86_64 >2014-08-10 14:05:07,688 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vte-0.28.2-13.fc20.x86_64. >2014-08-10 14:05:07,692 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: vte3-0.34.9-3.fc20.x86_64 >2014-08-10 14:05:07,704 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vte3-0.34.9-3.fc20.x86_64. >2014-08-10 14:05:07,705 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: xdg-utils-1.1.0-0.24.rc2.fc20.noarch >2014-08-10 14:05:07,713 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: xdg-utils-1.1.0-0.24.rc2.fc20.noarch. >2014-08-10 14:05:07,714 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: 1:xorg-x11-font-utils-7.5-18.fc20.x86_64 >2014-08-10 14:05:07,721 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: 1:xorg-x11-font-utils-7.5-18.fc20.x86_64. >2014-08-10 14:05:07,722 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: yajl-2.0.4-3.fc20.x86_64 >2014-08-10 14:05:07,729 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: yajl-2.0.4-3.fc20.x86_64. >2014-08-10 14:05:07,730 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: yum-utils-1.1.31-23.fc20.noarch >2014-08-10 14:05:07,738 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: yum-utils-1.1.31-23.fc20.noarch. >2014-08-10 14:05:07,738 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Download/Verify: zerofree-1.0.1-13.fc20.x86_64 >2014-08-10 14:05:07,746 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: zerofree-1.0.1-13.fc20.x86_64. >2014-08-10 14:05:07,746 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Status: Check Package Signatures >2014-08-10 14:05:07,754 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Status: Check Package Signatures. >2014-08-10 14:05:07,755 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Confirm use of GPG Key userid=oVirt <infra@ovirt.org> hexkeyid=FE590CB7 >2014-08-10 14:05:07,763 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Host h2 installation in progress . Confirm use of GPG Key userid=oVirt <infra@ovirt.org> hexkeyid=FE590CB7. >2014-08-10 14:05:07,959 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Status: Running Test Transaction >2014-08-10 14:05:07,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Status: Running Test Transaction. >2014-08-10 14:05:08,603 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Status: Running Transaction >2014-08-10 14:05:08,648 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Status: Running Transaction. >2014-08-10 14:05:08,992 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 1/331: 1:telnet-0.17-58.fc20.x86_64 >2014-08-10 14:05:09,013 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 1/331: 1:telnet-0.17-58.fc20.x86_64. >2014-08-10 14:05:09,267 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 2/331: atk-2.10.0-1.fc20.x86_64 >2014-08-10 14:05:09,278 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 2/331: atk-2.10.0-1.fc20.x86_64. >2014-08-10 14:05:09,675 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 3/331: glusterfs-libs-3.5.1-1.fc20.x86_64 >2014-08-10 14:05:09,683 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 3/331: glusterfs-libs-3.5.1-1.fc20.x86_64. >2014-08-10 14:05:09,975 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 4/331: libjpeg-turbo-1.3.1-2.fc20.x86_64 >2014-08-10 14:05:09,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 4/331: libjpeg-turbo-1.3.1-2.fc20.x86_64. >2014-08-10 14:05:10,276 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 5/331: yajl-2.0.4-3.fc20.x86_64 >2014-08-10 14:05:10,288 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 5/331: yajl-2.0.4-3.fc20.x86_64. >2014-08-10 14:05:10,547 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 6/331: libaio-0.3.109-8.fc20.x86_64 >2014-08-10 14:05:10,561 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 6/331: libaio-0.3.109-8.fc20.x86_64. >2014-08-10 14:05:10,818 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 7/331: numactl-libs-2.0.9-1.fc20.x86_64 >2014-08-10 14:05:10,834 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 7/331: numactl-libs-2.0.9-1.fc20.x86_64. >2014-08-10 14:05:11,103 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 8/331: libusbx-1.0.19-1.fc20.x86_64 >2014-08-10 14:05:11,149 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 8/331: libusbx-1.0.19-1.fc20.x86_64. >2014-08-10 14:05:11,416 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 9/331: glusterfs-3.5.1-1.fc20.x86_64 >2014-08-10 14:05:11,439 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 9/331: glusterfs-3.5.1-1.fc20.x86_64. >2014-08-10 14:05:11,915 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 10/331: glusterfs-api-3.5.1-1.fc20.x86_64 >2014-08-10 14:05:11,935 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 10/331: glusterfs-api-3.5.1-1.fc20.x86_64. >2014-08-10 14:05:12,257 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 11/331: libICE-1.0.8-6.fc20.x86_64 >2014-08-10 14:05:12,267 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 11/331: libICE-1.0.8-6.fc20.x86_64. >2014-08-10 14:05:12,598 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 12/331: fuse-libs-2.9.3-2.fc20.x86_64 >2014-08-10 14:05:12,606 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 12/331: fuse-libs-2.9.3-2.fc20.x86_64. >2014-08-10 14:05:12,938 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 13/331: libwsman1-2.4.3-1.fc20.x86_64 >2014-08-10 14:05:12,945 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 13/331: libwsman1-2.4.3-1.fc20.x86_64. >2014-08-10 14:05:13,292 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 14/331: 2:libogg-1.3.0-6.fc20.x86_64 >2014-08-10 14:05:13,301 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 14/331: 2:libogg-1.3.0-6.fc20.x86_64. >2014-08-10 14:05:13,613 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 15/331: libibverbs-1.1.7-3.fc20.x86_64 >2014-08-10 14:05:13,624 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 15/331: libibverbs-1.1.7-3.fc20.x86_64. >2014-08-10 14:05:13,959 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 16/331: libwayland-client-1.2.0-3.fc20.x86_64 >2014-08-10 14:05:13,980 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 16/331: libwayland-client-1.2.0-3.fc20.x86_64. >2014-08-10 14:05:14,290 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 17/331: perl-Data-Dumper-2.151-1.fc20.x86_64 >2014-08-10 14:05:14,303 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 17/331: perl-Data-Dumper-2.151-1.fc20.x86_64. >2014-08-10 14:05:14,644 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 18/331: pixman-0.30.0-4.fc20.x86_64 >2014-08-10 14:05:14,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 18/331: pixman-0.30.0-4.fc20.x86_64. >2014-08-10 14:05:15,010 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 19/331: librdmacm-1.0.17-2.fc20.x86_64 >2014-08-10 14:05:15,024 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 19/331: librdmacm-1.0.17-2.fc20.x86_64. >2014-08-10 14:05:15,348 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 20/331: libSM-1.2.1-6.fc20.x86_64 >2014-08-10 14:05:15,355 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 20/331: libSM-1.2.1-6.fc20.x86_64. >2014-08-10 14:05:15,679 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 21/331: sanlock-lib-3.0.0-2.fc20.x86_64 >2014-08-10 14:05:15,686 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 21/331: sanlock-lib-3.0.0-2.fc20.x86_64. >2014-08-10 14:05:15,972 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 22/331: mesa-libglapi-10.1.5-1.20140607.fc20.x86_64 >2014-08-10 14:05:15,992 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 22/331: mesa-libglapi-10.1.5-1.20140607.fc20.x86_64. >2014-08-10 14:05:16,254 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 23/331: libiscsi-1.9.0-4.fc20.x86_64 >2014-08-10 14:05:16,281 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 23/331: libiscsi-1.9.0-4.fc20.x86_64. >2014-08-10 14:05:16,555 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 24/331: perl-Compress-Raw-Zlib-2.062-2.fc20.x86_64 >2014-08-10 14:05:16,572 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 24/331: perl-Compress-Raw-Zlib-2.062-2.fc20.x86_64. >2014-08-10 14:05:16,834 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 25/331: 1:cups-libs-1.7.4-3.fc20.x86_64 >2014-08-10 14:05:16,845 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 25/331: 1:cups-libs-1.7.4-3.fc20.x86_64. >2014-08-10 14:05:17,181 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 26/331: augeas-libs-1.2.0-2.fc20.x86_64 >2014-08-10 14:05:17,203 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 26/331: augeas-libs-1.2.0-2.fc20.x86_64. >2014-08-10 14:05:17,568 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 27/331: python-cpopen-1.3-4.fc20.x86_64 >2014-08-10 14:05:17,582 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 27/331: python-cpopen-1.3-4.fc20.x86_64. >2014-08-10 14:05:17,843 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 28/331: bzip2-1.0.6-9.fc20.x86_64 >2014-08-10 14:05:17,855 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 28/331: bzip2-1.0.6-9.fc20.x86_64. >2014-08-10 14:05:18,172 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 29/331: libxslt-1.1.28-5.fc20.x86_64 >2014-08-10 14:05:18,178 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 29/331: libxslt-1.1.28-5.fc20.x86_64. >2014-08-10 14:05:18,910 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 30/331: sanlock-3.0.0-2.fc20.x86_64 >2014-08-10 14:05:18,956 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 30/331: sanlock-3.0.0-2.fc20.x86_64. >2014-08-10 14:05:19,253 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 31/331: glusterfs-fuse-3.5.1-1.fc20.x86_64 >2014-08-10 14:05:19,271 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 31/331: glusterfs-fuse-3.5.1-1.fc20.x86_64. >2014-08-10 14:05:19,548 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 32/331: jasper-libs-1.900.1-25.fc20.x86_64 >2014-08-10 14:05:19,561 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 32/331: jasper-libs-1.900.1-25.fc20.x86_64. >2014-08-10 14:05:20,360 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 33/331: libxml2-python-2.9.1-2.fc20.x86_64 >2014-08-10 14:05:20,455 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 33/331: libxml2-python-2.9.1-2.fc20.x86_64. >2014-08-10 14:05:20,707 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 34/331: hicolor-icon-theme-0.13-1.fc20.noarch >2014-08-10 14:05:20,720 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 34/331: hicolor-icon-theme-0.13-1.fc20.noarch. >2014-08-10 14:05:21,108 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 35/331: iptables-services-1.4.19.1-1.fc20.x86_64 >2014-08-10 14:05:21,126 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 35/331: iptables-services-1.4.19.1-1.fc20.x86_64. >2014-08-10 14:05:21,462 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 36/331: 1:perl-TimeDate-2.30-3.fc20.noarch >2014-08-10 14:05:21,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 36/331: 1:perl-TimeDate-2.30-3.fc20.noarch. >2014-08-10 14:05:21,741 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 37/331: perl-HTTP-Date-6.02-8.fc20.noarch >2014-08-10 14:05:21,755 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 37/331: perl-HTTP-Date-6.02-8.fc20.noarch. >2014-08-10 14:05:22,005 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 38/331: libtirpc-0.2.4-3.0.fc20.x86_64 >2014-08-10 14:05:22,012 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 38/331: libtirpc-0.2.4-3.0.fc20.x86_64. >2014-08-10 14:05:22,279 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 39/331: libfontenc-1.1.1-4.fc20.x86_64 >2014-08-10 14:05:22,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 39/331: libfontenc-1.1.1-4.fc20.x86_64. >2014-08-10 14:05:22,550 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 40/331: 1:net-snmp-libs-5.7.2-17.fc20.x86_64 >2014-08-10 14:05:22,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 40/331: 1:net-snmp-libs-5.7.2-17.fc20.x86_64. >2014-08-10 14:05:22,942 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 41/331: 1:net-snmp-utils-5.7.2-17.fc20.x86_64 >2014-08-10 14:05:22,955 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 41/331: 1:net-snmp-utils-5.7.2-17.fc20.x86_64. >2014-08-10 14:05:23,228 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 42/331: libwayland-server-1.2.0-3.fc20.x86_64 >2014-08-10 14:05:23,237 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 42/331: libwayland-server-1.2.0-3.fc20.x86_64. >2014-08-10 14:05:23,501 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 43/331: libqb-0.17.0-1.fc20.x86_64 >2014-08-10 14:05:23,510 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 43/331: libqb-0.17.0-1.fc20.x86_64. >2014-08-10 14:05:23,767 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 44/331: corosynclib-2.3.3-2.fc20.x86_64 >2014-08-10 14:05:23,775 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 44/331: corosynclib-2.3.3-2.fc20.x86_64. >2014-08-10 14:05:24,129 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 45/331: corosync-2.3.3-2.fc20.x86_64 >2014-08-10 14:05:24,139 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 45/331: corosync-2.3.3-2.fc20.x86_64. >2014-08-10 14:05:24,513 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 46/331: boost-system-1.54.0-9.fc20.x86_64 >2014-08-10 14:05:24,529 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 46/331: boost-system-1.54.0-9.fc20.x86_64. >2014-08-10 14:05:24,792 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 47/331: libcgroup-0.38-8.fc20.x86_64 >2014-08-10 14:05:24,802 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 47/331: libcgroup-0.38-8.fc20.x86_64. >2014-08-10 14:05:25,134 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 48/331: vdsm-python-zombiereaper-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:05:25,141 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 48/331: vdsm-python-zombiereaper-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:05:25,384 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 49/331: autogen-libopts-5.18-1.fc20.x86_64 >2014-08-10 14:05:25,398 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 49/331: autogen-libopts-5.18-1.fc20.x86_64. >2014-08-10 14:05:25,651 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 50/331: psmisc-22.20-3.fc20.x86_64 >2014-08-10 14:05:25,663 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 50/331: psmisc-22.20-3.fc20.x86_64. >2014-08-10 14:05:25,950 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 51/331: snappy-1.1.0-2.fc20.x86_64 >2014-08-10 14:05:25,969 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 51/331: snappy-1.1.0-2.fc20.x86_64. >2014-08-10 14:05:26,217 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 52/331: m2crypto-0.21.1-13.fc20.x86_64 >2014-08-10 14:05:26,226 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 52/331: m2crypto-0.21.1-13.fc20.x86_64. >2014-08-10 14:05:26,642 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 53/331: 2:ethtool-3.10-2.fc20.x86_64 >2014-08-10 14:05:26,789 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 53/331: 2:ethtool-3.10-2.fc20.x86_64. >2014-08-10 14:05:26,921 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 54/331: perl-Encode-Locale-1.03-7.fc20.noarch >2014-08-10 14:05:26,938 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 54/331: perl-Encode-Locale-1.03-7.fc20.noarch. >2014-08-10 14:05:27,171 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 55/331: 2:ntfs-3g-2014.2.15-1.fc20.x86_64 >2014-08-10 14:05:27,178 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 55/331: 2:ntfs-3g-2014.2.15-1.fc20.x86_64. >2014-08-10 14:05:27,500 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 56/331: libevent-2.0.21-3.fc20.x86_64 >2014-08-10 14:05:27,523 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 56/331: libevent-2.0.21-3.fc20.x86_64. >2014-08-10 14:05:27,809 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 57/331: python-suds-0.4.1-5.fc20.noarch >2014-08-10 14:05:27,824 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 57/331: python-suds-0.4.1-5.fc20.noarch. >2014-08-10 14:05:28,164 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 58/331: lcms2-2.6-1.fc20.x86_64 >2014-08-10 14:05:28,171 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 58/331: lcms2-2.6-1.fc20.x86_64. >2014-08-10 14:05:28,464 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 59/331: lzo-2.08-1.fc20.x86_64 >2014-08-10 14:05:28,478 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 59/331: lzo-2.08-1.fc20.x86_64. >2014-08-10 14:05:28,730 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 60/331: lzop-1.03-9.fc20.x86_64 >2014-08-10 14:05:28,743 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 60/331: lzop-1.03-9.fc20.x86_64. >2014-08-10 14:05:28,988 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 61/331: dosfstools-3.0.26-1.fc20.x86_64 >2014-08-10 14:05:28,999 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 61/331: dosfstools-3.0.26-1.fc20.x86_64. >2014-08-10 14:05:29,255 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 62/331: btrfs-progs-3.14.2-3.fc20.x86_64 >2014-08-10 14:05:29,264 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 62/331: btrfs-progs-3.14.2-3.fc20.x86_64. >2014-08-10 14:05:29,611 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 63/331: 2:ntfsprogs-2014.2.15-1.fc20.x86_64 >2014-08-10 14:05:29,629 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 63/331: 2:ntfsprogs-2014.2.15-1.fc20.x86_64. >2014-08-10 14:05:30,505 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 64/331: leveldb-1.12.0-5.fc20.x86_64 >2014-08-10 14:05:30,517 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 64/331: leveldb-1.12.0-5.fc20.x86_64. >2014-08-10 14:05:30,821 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 65/331: boost-thread-1.54.0-9.fc20.x86_64 >2014-08-10 14:05:30,883 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 65/331: boost-thread-1.54.0-9.fc20.x86_64. >2014-08-10 14:05:31,079 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 66/331: sheepdog-0.3.0-5.fc20.x86_64 >2014-08-10 14:05:31,097 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 66/331: sheepdog-0.3.0-5.fc20.x86_64. >2014-08-10 14:05:31,414 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 67/331: mesa-libgbm-10.1.5-1.20140607.fc20.x86_64 >2014-08-10 14:05:31,428 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 67/331: mesa-libgbm-10.1.5-1.20140607.fc20.x86_64. >2014-08-10 14:05:31,681 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 68/331: libXfont-1.4.8-1.fc20.x86_64 >2014-08-10 14:05:31,706 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 68/331: libXfont-1.4.8-1.fc20.x86_64. >2014-08-10 14:05:31,965 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 69/331: 1:xorg-x11-font-utils-7.5-18.fc20.x86_64 >2014-08-10 14:05:31,983 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 69/331: 1:xorg-x11-font-utils-7.5-18.fc20.x86_64. >2014-08-10 14:05:32,582 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 70/331: rpcbind-0.2.1-0.2.fc20.x86_64 >2014-08-10 14:05:32,628 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 70/331: rpcbind-0.2.1-0.2.fc20.x86_64. >2014-08-10 14:05:32,969 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 71/331: perl-File-Listing-6.04-7.fc20.noarch >2014-08-10 14:05:32,984 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 71/331: perl-File-Listing-6.04-7.fc20.noarch. >2014-08-10 14:05:33,257 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 72/331: sos-3.1-1.1.fc20.ovirt.noarch >2014-08-10 14:05:33,266 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 72/331: sos-3.1-1.1.fc20.ovirt.noarch. >2014-08-10 14:05:33,885 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 73/331: python-dmidecode-3.10.13-10.fc20.x86_64 >2014-08-10 14:05:33,937 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 73/331: python-dmidecode-3.10.13-10.fc20.x86_64. >2014-08-10 14:05:34,164 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 74/331: netcf-libs-0.2.3-6.fc20.x86_64 >2014-08-10 14:05:34,177 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 74/331: netcf-libs-0.2.3-6.fc20.x86_64. >2014-08-10 14:05:34,475 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 75/331: libosinfo-0.2.9-1.fc20.x86_64 >2014-08-10 14:05:34,483 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 75/331: libosinfo-0.2.9-1.fc20.x86_64. >2014-08-10 14:05:34,824 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 76/331: augeas-1.2.0-2.fc20.x86_64 >2014-08-10 14:05:34,831 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 76/331: augeas-1.2.0-2.fc20.x86_64. >2014-08-10 14:05:35,962 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 77/331: sanlock-python-3.0.0-2.fc20.x86_64 >2014-08-10 14:05:35,973 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 77/331: sanlock-python-3.0.0-2.fc20.x86_64. >2014-08-10 14:05:36,216 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 78/331: glusterfs-rdma-3.5.1-1.fc20.x86_64 >2014-08-10 14:05:36,222 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 78/331: glusterfs-rdma-3.5.1-1.fc20.x86_64. >2014-08-10 14:05:36,466 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 79/331: libwayland-cursor-1.2.0-3.fc20.x86_64 >2014-08-10 14:05:36,487 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 79/331: libwayland-cursor-1.2.0-3.fc20.x86_64. >2014-08-10 14:05:36,771 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 80/331: flac-libs-1.3.0-3.fc20.x86_64 >2014-08-10 14:05:36,785 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 80/331: flac-libs-1.3.0-3.fc20.x86_64. >2014-08-10 14:05:37,092 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 81/331: 1:libvorbis-1.3.4-1.fc20.x86_64 >2014-08-10 14:05:37,099 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 81/331: 1:libvorbis-1.3.4-1.fc20.x86_64. >2014-08-10 14:05:37,408 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 82/331: celt051-0.5.1.3-7.fc20.x86_64 >2014-08-10 14:05:37,414 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 82/331: celt051-0.5.1.3-7.fc20.x86_64. >2014-08-10 14:05:37,841 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 83/331: uml_utilities-20070815-12.fc20.x86_64 >2014-08-10 14:05:37,861 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 83/331: uml_utilities-20070815-12.fc20.x86_64. >2014-08-10 14:05:38,140 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 84/331: usbredir-0.6-5.fc20.x86_64 >2014-08-10 14:05:38,151 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 84/331: usbredir-0.6-5.fc20.x86_64. >2014-08-10 14:05:38,415 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 85/331: libgusb-0.1.6-2.fc20.x86_64 >2014-08-10 14:05:38,424 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 85/331: libgusb-0.1.6-2.fc20.x86_64. >2014-08-10 14:05:38,689 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 86/331: colord-libs-1.1.7-1.fc20.x86_64 >2014-08-10 14:05:38,697 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 86/331: colord-libs-1.1.7-1.fc20.x86_64. >2014-08-10 14:05:38,976 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 87/331: 1:libusb-0.1.5-3.fc20.x86_64 >2014-08-10 14:05:38,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 87/331: 1:libusb-0.1.5-3.fc20.x86_64. >2014-08-10 14:05:39,263 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 88/331: gnupg-1.4.18-1.fc20.x86_64 >2014-08-10 14:05:39,277 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 88/331: gnupg-1.4.18-1.fc20.x86_64. >2014-08-10 14:05:39,777 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 89/331: numactl-2.0.9-1.fc20.x86_64 >2014-08-10 14:05:39,799 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 89/331: numactl-2.0.9-1.fc20.x86_64. >2014-08-10 14:05:40,189 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 90/331: device-mapper-multipath-libs-0.4.9-56.fc20.x86_64 >2014-08-10 14:05:40,196 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 90/331: device-mapper-multipath-libs-0.4.9-56.fc20.x86_64. >2014-08-10 14:05:40,523 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 91/331: device-mapper-multipath-0.4.9-56.fc20.x86_64 >2014-08-10 14:05:40,535 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 91/331: device-mapper-multipath-0.4.9-56.fc20.x86_64. >2014-08-10 14:05:40,856 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 92/331: ioprocess-0.5.0-1.fc20.x86_64 >2014-08-10 14:05:40,867 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 92/331: ioprocess-0.5.0-1.fc20.x86_64. >2014-08-10 14:05:41,104 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 93/331: python-ioprocess-0.5.0-1.fc20.noarch >2014-08-10 14:05:41,115 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 93/331: python-ioprocess-0.5.0-1.fc20.noarch. >2014-08-10 14:05:41,356 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 94/331: vdsm-python-4.16.1-6.gita4a4614.fc20.x86_64 >2014-08-10 14:05:41,363 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 94/331: vdsm-python-4.16.1-6.gita4a4614.fc20.x86_64. >2014-08-10 14:05:41,702 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 95/331: vdsm-xmlrpc-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:05:41,745 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 95/331: vdsm-xmlrpc-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:05:41,960 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 96/331: glusterfs-cli-3.5.1-1.fc20.x86_64 >2014-08-10 14:05:41,976 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 96/331: glusterfs-cli-3.5.1-1.fc20.x86_64. >2014-08-10 14:05:42,251 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 97/331: hivex-1.3.8-3.fc20.x86_64 >2014-08-10 14:05:42,257 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 97/331: hivex-1.3.8-3.fc20.x86_64. >2014-08-10 14:05:42,658 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 98/331: perl-hivex-1.3.8-3.fc20.x86_64 >2014-08-10 14:05:42,804 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 98/331: perl-hivex-1.3.8-3.fc20.x86_64. >2014-08-10 14:05:42,938 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 99/331: python-IPy-0.75-6.fc20.noarch >2014-08-10 14:05:42,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 99/331: python-IPy-0.75-6.fc20.noarch. >2014-08-10 14:05:43,181 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 100/331: perl-IO-HTML-1.001-1.fc20.noarch >2014-08-10 14:05:43,201 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 100/331: perl-IO-HTML-1.001-1.fc20.noarch. >2014-08-10 14:05:43,438 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 101/331: libthai-0.1.20-1.fc20.x86_64 >2014-08-10 14:05:43,450 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 101/331: libthai-0.1.20-1.fc20.x86_64. >2014-08-10 14:05:43,740 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 102/331: perl-HTML-Tagset-3.20-18.fc20.noarch >2014-08-10 14:05:43,748 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 102/331: perl-HTML-Tagset-3.20-18.fc20.noarch. >2014-08-10 14:05:43,995 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 103/331: perl-IO-Socket-IP-0.30-2.fc20.noarch >2014-08-10 14:05:44,004 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 103/331: perl-IO-Socket-IP-0.30-2.fc20.noarch. >2014-08-10 14:05:44,262 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 104/331: libnfsidmap-0.25-8.fc20.x86_64 >2014-08-10 14:05:44,269 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 104/331: libnfsidmap-0.25-8.fc20.x86_64. >2014-08-10 14:05:44,567 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 105/331: attr-2.4.47-3.fc20.x86_64 >2014-08-10 14:05:44,576 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 105/331: attr-2.4.47-3.fc20.x86_64. >2014-08-10 14:05:44,835 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 106/331: fuse-2.9.3-2.fc20.x86_64 >2014-08-10 14:05:44,849 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 106/331: fuse-2.9.3-2.fc20.x86_64. >2014-08-10 14:05:45,282 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 107/331: 2:nmap-ncat-6.45-1.fc20.x86_64 >2014-08-10 14:05:45,395 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 107/331: 2:nmap-ncat-6.45-1.fc20.x86_64. >2014-08-10 14:05:45,563 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 108/331: pm-utils-1.4.1-26.fc20.x86_64 >2014-08-10 14:05:45,578 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 108/331: pm-utils-1.4.1-26.fc20.x86_64. >2014-08-10 14:05:45,894 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 109/331: 1:pyparted-3.9-4.fc20.x86_64 >2014-08-10 14:05:45,909 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 109/331: 1:pyparted-3.9-4.fc20.x86_64. >2014-08-10 14:05:46,197 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 110/331: mtools-4.0.18-4.fc20.x86_64 >2014-08-10 14:05:46,207 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 110/331: mtools-4.0.18-4.fc20.x86_64. >2014-08-10 14:05:46,541 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 111/331: syslinux-4.05-7.fc20.x86_64 >2014-08-10 14:05:46,555 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 111/331: syslinux-4.05-7.fc20.x86_64. >2014-08-10 14:05:46,962 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 112/331: syslinux-extlinux-4.05-7.fc20.x86_64 >2014-08-10 14:05:46,985 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 112/331: syslinux-extlinux-4.05-7.fc20.x86_64. >2014-08-10 14:05:47,305 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 113/331: cryptopp-5.6.2-3.fc20.x86_64 >2014-08-10 14:05:47,316 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 113/331: cryptopp-5.6.2-3.fc20.x86_64. >2014-08-10 14:05:47,715 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 114/331: ceph-libs-0.81.0-5.fc20.x86_64 >2014-08-10 14:05:47,863 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 114/331: ceph-libs-0.81.0-5.fc20.x86_64. >2014-08-10 14:05:48,250 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 115/331: 2:qemu-img-1.6.2-7.fc20.x86_64 >2014-08-10 14:05:48,272 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 115/331: 2:qemu-img-1.6.2-7.fc20.x86_64. >2014-08-10 14:05:48,571 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 116/331: 2:qemu-common-1.6.2-7.fc20.x86_64 >2014-08-10 14:05:48,591 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 116/331: 2:qemu-common-1.6.2-7.fc20.x86_64. >2014-08-10 14:05:49,447 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 117/331: seavgabios-bin-1.7.3.1-2.fc20.noarch >2014-08-10 14:05:49,486 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 117/331: seavgabios-bin-1.7.3.1-2.fc20.noarch. >2014-08-10 14:05:49,690 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 118/331: perl-Convert-BinHex-1.122-1.fc20.noarch >2014-08-10 14:05:49,701 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 118/331: perl-Convert-BinHex-1.122-1.fc20.noarch. >2014-08-10 14:05:49,963 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 119/331: libusal-1.1.11-22.fc20.x86_64 >2014-08-10 14:05:49,974 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 119/331: libusal-1.1.11-22.fc20.x86_64. >2014-08-10 14:05:50,550 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 120/331: genisoimage-1.1.11-22.fc20.x86_64 >2014-08-10 14:05:50,562 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 120/331: genisoimage-1.1.11-22.fc20.x86_64. >2014-08-10 14:05:50,983 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 121/331: python-ethtool-0.10-1.fc20.x86_64 >2014-08-10 14:05:51,026 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 121/331: python-ethtool-0.10-1.fc20.x86_64. >2014-08-10 14:05:51,240 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 122/331: hexedit-1.2.13-4.fc20.x86_64 >2014-08-10 14:05:51,249 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 122/331: hexedit-1.2.13-4.fc20.x86_64. >2014-08-10 14:05:51,508 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 123/331: 1:quota-nls-4.01-11.fc20.noarch >2014-08-10 14:05:51,514 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 123/331: 1:quota-nls-4.01-11.fc20.noarch. >2014-08-10 14:05:51,772 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 124/331: poppler-data-0.4.6-4.fc20.noarch >2014-08-10 14:05:51,779 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 124/331: poppler-data-0.4.6-4.fc20.noarch. >2014-08-10 14:05:52,516 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 125/331: gsm-1.0.13-10.fc20.x86_64 >2014-08-10 14:05:52,591 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 125/331: gsm-1.0.13-10.fc20.x86_64. >2014-08-10 14:05:52,798 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 126/331: libsndfile-1.0.25-8.fc20.x86_64 >2014-08-10 14:05:52,814 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 126/331: libsndfile-1.0.25-8.fc20.x86_64. >2014-08-10 14:05:53,095 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 127/331: mailcap-2.1.42-1.fc20.noarch >2014-08-10 14:05:53,104 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 127/331: mailcap-2.1.42-1.fc20.noarch. >2014-08-10 14:05:53,350 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 128/331: perl-LWP-MediaTypes-6.02-4.fc20.noarch >2014-08-10 14:05:53,385 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 128/331: perl-LWP-MediaTypes-6.02-4.fc20.noarch. >2014-08-10 14:05:53,606 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 129/331: libnl-1.1.4-3.fc20.x86_64 >2014-08-10 14:05:53,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 129/331: libnl-1.1.4-3.fc20.x86_64. >2014-08-10 14:05:53,875 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 130/331: libseccomp-2.1.0-1.fc20.x86_64 >2014-08-10 14:05:53,882 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 130/331: libseccomp-2.1.0-1.fc20.x86_64. >2014-08-10 14:05:54,148 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 131/331: tcp_wrappers-7.6-76.fc20.x86_64 >2014-08-10 14:05:54,155 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 131/331: tcp_wrappers-7.6-76.fc20.x86_64. >2014-08-10 14:05:54,400 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 132/331: 1:quota-4.01-11.fc20.x86_64 >2014-08-10 14:05:54,412 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 132/331: 1:quota-4.01-11.fc20.x86_64. >2014-08-10 14:05:54,798 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 133/331: perl-Net-LibIDN-0.12-16.fc20.x86_64 >2014-08-10 14:05:54,809 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 133/331: perl-Net-LibIDN-0.12-16.fc20.x86_64. >2014-08-10 14:05:55,041 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 134/331: python-inotify-0.9.4-4.fc20.noarch >2014-08-10 14:05:55,050 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 134/331: python-inotify-0.9.4-4.fc20.noarch. >2014-08-10 14:05:55,739 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 135/331: zerofree-1.0.1-13.fc20.x86_64 >2014-08-10 14:05:55,795 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 135/331: zerofree-1.0.1-13.fc20.x86_64. >2014-08-10 14:05:56,258 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 136/331: ldns-1.6.16-6.fc20.x86_64 >2014-08-10 14:05:56,333 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 136/331: ldns-1.6.16-6.fc20.x86_64. >2014-08-10 14:05:57,101 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 137/331: unbound-libs-1.4.21-3.fc20.x86_64 >2014-08-10 14:05:57,153 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 137/331: unbound-libs-1.4.21-3.fc20.x86_64. >2014-08-10 14:05:58,036 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 138/331: gnutls-dane-3.1.25-1.fc20.x86_64 >2014-08-10 14:05:58,080 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 138/331: gnutls-dane-3.1.25-1.fc20.x86_64. >2014-08-10 14:05:58,362 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 139/331: gnutls-utils-3.1.25-1.fc20.x86_64 >2014-08-10 14:05:58,370 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 139/331: gnutls-utils-3.1.25-1.fc20.x86_64. >2014-08-10 14:05:58,730 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 140/331: python-backports-1.0-3.fc20.x86_64 >2014-08-10 14:05:58,755 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 140/331: python-backports-1.0-3.fc20.x86_64. >2014-08-10 14:05:59,008 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 141/331: python-backports-ssl_match_hostname-3.4.0.2-1.fc20.noarch >2014-08-10 14:05:59,024 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 141/331: python-backports-ssl_match_hostname-3.4.0.2-1.fc20.noarch. >2014-08-10 14:05:59,289 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 142/331: python-setuptools-1.4.2-1.fc20.noarch >2014-08-10 14:05:59,297 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 142/331: python-setuptools-1.4.2-1.fc20.noarch. >2014-08-10 14:05:59,770 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 143/331: python-pyudev-0.15-5.fc20.noarch >2014-08-10 14:05:59,844 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 143/331: python-pyudev-0.15-5.fc20.noarch. >2014-08-10 14:06:00,071 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 144/331: numad-0.5-12.20130814git.fc20.x86_64 >2014-08-10 14:06:00,084 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 144/331: numad-0.5-12.20130814git.fc20.x86_64. >2014-08-10 14:06:00,369 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 145/331: alsa-lib-1.0.27.2-2.fc20.x86_64 >2014-08-10 14:06:00,390 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 145/331: alsa-lib-1.0.27.2-2.fc20.x86_64. >2014-08-10 14:06:00,754 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 146/331: perl-Digest-1.17-291.fc20.noarch >2014-08-10 14:06:00,763 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 146/331: perl-Digest-1.17-291.fc20.noarch. >2014-08-10 14:06:01,282 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 147/331: perl-Digest-MD5-2.53-3.fc20.x86_64 >2014-08-10 14:06:01,301 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 147/331: perl-Digest-MD5-2.53-3.fc20.x86_64. >2014-08-10 14:06:01,525 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 148/331: perl-XML-Parser-2.41-11.fc20.x86_64 >2014-08-10 14:06:01,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 148/331: perl-XML-Parser-2.41-11.fc20.x86_64. >2014-08-10 14:06:01,844 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 149/331: libicu-50.1.2-10.fc20.x86_64 >2014-08-10 14:06:01,856 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 149/331: libicu-50.1.2-10.fc20.x86_64. >2014-08-10 14:06:02,873 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 150/331: logrotate-3.8.7-1.fc20.x86_64 >2014-08-10 14:06:02,924 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 150/331: logrotate-3.8.7-1.fc20.x86_64. >2014-08-10 14:06:03,137 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 151/331: virt-what-1.13-2.fc20.x86_64 >2014-08-10 14:06:03,148 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 151/331: virt-what-1.13-2.fc20.x86_64. >2014-08-10 14:06:03,382 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 152/331: 1:emacs-filesystem-24.3-17.fc20.noarch >2014-08-10 14:06:03,396 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 152/331: 1:emacs-filesystem-24.3-17.fc20.noarch. >2014-08-10 14:06:03,613 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 153/331: desktop-file-utils-0.22-1.fc20.x86_64 >2014-08-10 14:06:03,619 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 153/331: desktop-file-utils-0.22-1.fc20.x86_64. >2014-08-10 14:06:03,911 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 154/331: xdg-utils-1.1.0-0.24.rc2.fc20.noarch >2014-08-10 14:06:03,926 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 154/331: xdg-utils-1.1.0-0.24.rc2.fc20.noarch. >2014-08-10 14:06:04,163 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 155/331: opus-1.1-1.fc20.x86_64 >2014-08-10 14:06:04,174 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 155/331: opus-1.1-1.fc20.x86_64. >2014-08-10 14:06:04,502 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 156/331: spice-server-0.12.5-2.fc20.x86_64 >2014-08-10 14:06:04,514 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 156/331: spice-server-0.12.5-2.fc20.x86_64. >2014-08-10 14:06:04,865 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 157/331: python-configobj-4.7.2-7.fc20.noarch >2014-08-10 14:06:04,878 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 157/331: python-configobj-4.7.2-7.fc20.noarch. >2014-08-10 14:06:05,410 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 158/331: radvd-1.9.9-1.fc20.x86_64 >2014-08-10 14:06:05,457 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 158/331: radvd-1.9.9-1.fc20.x86_64. >2014-08-10 14:06:05,714 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 159/331: libsemanage-python-2.1.10-14.fc20.x86_64 >2014-08-10 14:06:05,722 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 159/331: libsemanage-python-2.1.10-14.fc20.x86_64. >2014-08-10 14:06:06,206 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 160/331: seabios-bin-1.7.3.1-2.fc20.noarch >2014-08-10 14:06:06,261 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 160/331: seabios-bin-1.7.3.1-2.fc20.noarch. >2014-08-10 14:06:06,447 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 161/331: perl-Compress-Raw-Bzip2-2.062-2.fc20.x86_64 >2014-08-10 14:06:06,459 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 161/331: perl-Compress-Raw-Bzip2-2.062-2.fc20.x86_64. >2014-08-10 14:06:06,690 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 162/331: perl-IO-Compress-2.062-2.fc20.noarch >2014-08-10 14:06:06,699 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 162/331: perl-IO-Compress-2.062-2.fc20.noarch. >2014-08-10 14:06:07,012 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 163/331: perl-Email-Date-Format-1.002-17.fc20.noarch >2014-08-10 14:06:07,022 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 163/331: perl-Email-Date-Format-1.002-17.fc20.noarch. >2014-08-10 14:06:07,282 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 164/331: SDL-1.2.15-13.fc20.x86_64 >2014-08-10 14:06:07,295 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 164/331: SDL-1.2.15-13.fc20.x86_64. >2014-08-10 14:06:07,592 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 165/331: python-pexpect-3.1-1.fc20.noarch >2014-08-10 14:06:07,602 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 165/331: python-pexpect-3.1-1.fc20.noarch. >2014-08-10 14:06:07,918 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 166/331: fence-agents-common-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:07,925 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 166/331: fence-agents-common-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:08,170 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 167/331: fence-agents-rhevm-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:08,181 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 167/331: fence-agents-rhevm-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:08,509 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 168/331: fence-agents-apc-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:08,521 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 168/331: fence-agents-apc-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:08,767 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 169/331: fence-agents-kdump-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:08,778 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 169/331: fence-agents-kdump-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:09,030 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 170/331: fence-agents-alom-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:09,055 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 170/331: fence-agents-alom-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:09,287 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 171/331: fence-agents-lpar-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:09,307 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 171/331: fence-agents-lpar-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:09,544 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 172/331: fence-agents-wti-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:09,556 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 172/331: fence-agents-wti-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:09,832 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 173/331: fence-agents-ovh-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:09,859 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 173/331: fence-agents-ovh-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:10,114 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 174/331: fence-agents-netio-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:10,127 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 174/331: fence-agents-netio-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:10,385 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 175/331: fence-agents-ifmib-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:10,400 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 175/331: fence-agents-ifmib-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:10,679 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 176/331: fence-agents-ipdu-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:10,723 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 176/331: fence-agents-ipdu-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:11,132 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 177/331: fence-agents-eaton-snmp-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:11,261 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 177/331: fence-agents-eaton-snmp-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:11,412 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 178/331: fence-agents-eps-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:11,419 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 178/331: fence-agents-eps-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:11,692 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 179/331: fence-agents-hpblade-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:11,700 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 179/331: fence-agents-hpblade-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:11,974 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 180/331: fence-agents-cisco-ucs-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:11,982 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 180/331: fence-agents-cisco-ucs-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:12,254 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 181/331: fence-agents-intelmodular-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:12,263 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 181/331: fence-agents-intelmodular-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:12,538 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 182/331: fence-agents-docker-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:12,545 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 182/331: fence-agents-docker-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:12,793 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 183/331: fence-agents-vmware-soap-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:12,801 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 183/331: fence-agents-vmware-soap-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:13,049 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 184/331: fence-agents-ilo-mp-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:13,058 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 184/331: fence-agents-ilo-mp-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:13,304 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 185/331: fence-agents-bladecenter-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:13,315 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 185/331: fence-agents-bladecenter-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:13,639 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 186/331: fence-agents-ldom-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:13,646 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 186/331: fence-agents-ldom-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:13,894 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 187/331: fence-agents-cisco-mds-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:13,902 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 187/331: fence-agents-cisco-mds-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:14,153 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 188/331: fence-agents-drac5-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:14,192 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 188/331: fence-agents-drac5-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:14,408 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 189/331: fence-agents-rsb-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:14,432 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 189/331: fence-agents-rsb-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:14,685 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 190/331: fence-agents-rsa-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:14,747 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 190/331: fence-agents-rsa-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:14,940 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 191/331: fence-agents-apc-snmp-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:14,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 191/331: fence-agents-apc-snmp-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:15,197 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 192/331: fence-agents-ibmblade-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:15,211 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 192/331: fence-agents-ibmblade-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:15,454 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 193/331: fence-agents-ilo2-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:15,467 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 193/331: fence-agents-ilo2-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:15,710 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 194/331: fence-agents-brocade-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:15,757 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 194/331: fence-agents-brocade-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:16,065 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 195/331: sg3_utils-libs-1.37-2.fc20.x86_64 >2014-08-10 14:06:16,072 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 195/331: sg3_utils-libs-1.37-2.fc20.x86_64. >2014-08-10 14:06:16,338 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 196/331: sg3_utils-1.37-2.fc20.x86_64 >2014-08-10 14:06:16,345 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 196/331: sg3_utils-1.37-2.fc20.x86_64. >2014-08-10 14:06:16,722 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 197/331: fence-agents-scsi-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:16,734 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 197/331: fence-agents-scsi-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:16,978 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 198/331: bluez-libs-5.18-1.fc20.x86_64 >2014-08-10 14:06:16,991 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 198/331: bluez-libs-5.18-1.fc20.x86_64. >2014-08-10 14:06:17,277 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 199/331: setools-libs-3.3.7-41.fc20.x86_64 >2014-08-10 14:06:17,289 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 199/331: setools-libs-3.3.7-41.fc20.x86_64. >2014-08-10 14:06:17,621 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 200/331: lsscsi-0.27-2.fc20.x86_64 >2014-08-10 14:06:17,628 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 200/331: lsscsi-0.27-2.fc20.x86_64. >2014-08-10 14:06:17,864 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 201/331: 1:sgabios-bin-0.20110622svn-6.fc20.noarch >2014-08-10 14:06:17,877 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 201/331: 1:sgabios-bin-0.20110622svn-6.fc20.noarch. >2014-08-10 14:06:18,094 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 202/331: perl-MIME-Types-2.04-1.fc20.noarch >2014-08-10 14:06:18,108 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 202/331: perl-MIME-Types-2.04-1.fc20.noarch. >2014-08-10 14:06:18,346 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 203/331: perl-MIME-Lite-3.030-1.fc20.noarch >2014-08-10 14:06:18,357 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 203/331: perl-MIME-Lite-3.030-1.fc20.noarch. >2014-08-10 14:06:18,702 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 204/331: json-glib-0.16.2-1.fc20.x86_64 >2014-08-10 14:06:18,713 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 204/331: json-glib-0.16.2-1.fc20.x86_64. >2014-08-10 14:06:19,056 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 205/331: libldm-0.2.3-4.fc20.x86_64 >2014-08-10 14:06:19,069 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 205/331: libldm-0.2.3-4.fc20.x86_64. >2014-08-10 14:06:19,336 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 206/331: python-pthreading-0.1.3-3.fc20.noarch >2014-08-10 14:06:19,363 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 206/331: python-pthreading-0.1.3-3.fc20.noarch. >2014-08-10 14:06:19,600 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 207/331: libX11-common-1.6.1-1.fc20.noarch >2014-08-10 14:06:19,615 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 207/331: libX11-common-1.6.1-1.fc20.noarch. >2014-08-10 14:06:19,976 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 208/331: checkpolicy-2.1.12-5.fc20.x86_64 >2014-08-10 14:06:19,996 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 208/331: checkpolicy-2.1.12-5.fc20.x86_64. >2014-08-10 14:06:20,261 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 209/331: ipxe-roms-qemu-20140303-1.gitff1e7fc7.fc20.noarch >2014-08-10 14:06:20,269 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 209/331: ipxe-roms-qemu-20140303-1.gitff1e7fc7.fc20.noarch. >2014-08-10 14:06:20,589 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 210/331: scrub-2.5.2-4.fc20.x86_64 >2014-08-10 14:06:20,617 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 210/331: scrub-2.5.2-4.fc20.x86_64. >2014-08-10 14:06:21,781 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 211/331: cyrus-sasl-2.1.26-14.fc20.x86_64 >2014-08-10 14:06:21,826 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 211/331: cyrus-sasl-2.1.26-14.fc20.x86_64. >2014-08-10 14:06:22,117 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 212/331: jbigkit-libs-2.0-10.fc20.x86_64 >2014-08-10 14:06:22,124 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 212/331: jbigkit-libs-2.0-10.fc20.x86_64. >2014-08-10 14:06:22,369 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 213/331: libtiff-4.0.3-15.fc20.x86_64 >2014-08-10 14:06:22,381 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 213/331: libtiff-4.0.3-15.fc20.x86_64. >2014-08-10 14:06:22,684 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 214/331: perl-Business-ISBN-Data-20120719.001-4.fc20.noarch >2014-08-10 14:06:22,695 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 214/331: perl-Business-ISBN-Data-20120719.001-4.fc20.noarch. >2014-08-10 14:06:22,954 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 215/331: perl-Business-ISBN-2.06-4.fc20.noarch >2014-08-10 14:06:22,960 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 215/331: perl-Business-ISBN-2.06-4.fc20.noarch. >2014-08-10 14:06:23,198 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 216/331: perl-URI-1.60-11.fc20.noarch >2014-08-10 14:06:23,209 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 216/331: perl-URI-1.60-11.fc20.noarch. >2014-08-10 14:06:23,539 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 217/331: perl-HTTP-Message-6.06-7.fc20.noarch >2014-08-10 14:06:23,548 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 217/331: perl-HTTP-Message-6.06-7.fc20.noarch. >2014-08-10 14:06:23,873 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 218/331: perl-HTTP-Daemon-6.01-7.fc20.noarch >2014-08-10 14:06:23,879 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 218/331: perl-HTTP-Daemon-6.01-7.fc20.noarch. >2014-08-10 14:06:24,172 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 219/331: perl-HTML-Parser-3.71-4.fc20.x86_64 >2014-08-10 14:06:24,186 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 219/331: perl-HTML-Parser-3.71-4.fc20.x86_64. >2014-08-10 14:06:24,685 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 220/331: perl-HTTP-Cookies-6.01-7.fc20.noarch >2014-08-10 14:06:24,753 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 220/331: perl-HTTP-Cookies-6.01-7.fc20.noarch. >2014-08-10 14:06:25,006 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 221/331: perl-HTTP-Negotiate-6.01-7.fc20.noarch >2014-08-10 14:06:25,030 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 221/331: perl-HTTP-Negotiate-6.01-7.fc20.noarch. >2014-08-10 14:06:25,311 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 222/331: perl-WWW-RobotRules-6.02-8.fc20.noarch >2014-08-10 14:06:25,320 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 222/331: perl-WWW-RobotRules-6.02-8.fc20.noarch. >2014-08-10 14:06:25,634 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 223/331: python-chardet-2.0.1-7.fc20.noarch >2014-08-10 14:06:25,643 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 223/331: python-chardet-2.0.1-7.fc20.noarch. >2014-08-10 14:06:26,063 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 224/331: python-kitchen-1.1.1-5.fc20.noarch >2014-08-10 14:06:26,173 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 224/331: python-kitchen-1.1.1-5.fc20.noarch. >2014-08-10 14:06:27,343 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 225/331: yum-utils-1.1.31-23.fc20.noarch >2014-08-10 14:06:27,407 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 225/331: yum-utils-1.1.31-23.fc20.noarch. >2014-08-10 14:06:27,913 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 226/331: supermin-5.1.9-1.fc20.x86_64 >2014-08-10 14:06:27,978 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 226/331: supermin-5.1.9-1.fc20.x86_64. >2014-08-10 14:06:28,342 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 227/331: libxkbcommon-0.3.1-1.fc20.x86_64 >2014-08-10 14:06:28,351 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 227/331: libxkbcommon-0.3.1-1.fc20.x86_64. >2014-08-10 14:06:28,963 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 228/331: ntpdate-4.2.6p5-18.fc20.x86_64 >2014-08-10 14:06:28,997 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 228/331: ntpdate-4.2.6p5-18.fc20.x86_64. >2014-08-10 14:06:29,399 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 229/331: ntp-4.2.6p5-18.fc20.x86_64 >2014-08-10 14:06:29,419 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 229/331: ntp-4.2.6p5-18.fc20.x86_64. >2014-08-10 14:06:30,062 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 230/331: cryptsetup-1.6.4-1.fc20.x86_64 >2014-08-10 14:06:30,082 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 230/331: cryptsetup-1.6.4-1.fc20.x86_64. >2014-08-10 14:06:30,549 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 231/331: json-c-0.11-6.fc20.x86_64 >2014-08-10 14:06:30,580 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 231/331: json-c-0.11-6.fc20.x86_64. >2014-08-10 14:06:30,843 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 232/331: perl-Net-SSLeay-1.55-4.fc20.x86_64 >2014-08-10 14:06:30,853 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 232/331: perl-Net-SSLeay-1.55-4.fc20.x86_64. >2014-08-10 14:06:31,231 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 233/331: perl-IO-Socket-SSL-1.955-2.fc20.noarch >2014-08-10 14:06:31,243 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 233/331: perl-IO-Socket-SSL-1.955-2.fc20.noarch. >2014-08-10 14:06:31,562 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 234/331: perl-Net-HTTP-6.06-4.fc20.noarch >2014-08-10 14:06:31,574 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 234/331: perl-Net-HTTP-6.06-4.fc20.noarch. >2014-08-10 14:06:32,162 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 235/331: perl-libwww-perl-6.05-3.fc20.noarch >2014-08-10 14:06:32,211 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 235/331: perl-libwww-perl-6.05-3.fc20.noarch. >2014-08-10 14:06:32,595 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 236/331: icoutils-0.31.0-3.fc20.x86_64 >2014-08-10 14:06:32,634 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 236/331: icoutils-0.31.0-3.fc20.x86_64. >2014-08-10 14:06:32,897 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 237/331: perl-Net-SMTP-SSL-1.01-16.fc20.noarch >2014-08-10 14:06:32,907 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 237/331: perl-Net-SMTP-SSL-1.01-16.fc20.noarch. >2014-08-10 14:06:33,173 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 238/331: perl-MailTools-2.12-4.fc20.noarch >2014-08-10 14:06:33,180 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 238/331: perl-MailTools-2.12-4.fc20.noarch. >2014-08-10 14:06:33,575 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 239/331: perl-MIME-tools-5.505-1.fc20.noarch >2014-08-10 14:06:33,586 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 239/331: perl-MIME-tools-5.505-1.fc20.noarch. >2014-08-10 14:06:33,901 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 240/331: graphite2-1.2.2-4.fc20.x86_64 >2014-08-10 14:06:33,909 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 240/331: graphite2-1.2.2-4.fc20.x86_64. >2014-08-10 14:06:34,190 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 241/331: harfbuzz-0.9.27-1.fc20.x86_64 >2014-08-10 14:06:34,198 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 241/331: harfbuzz-0.9.27-1.fc20.x86_64. >2014-08-10 14:06:34,480 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 242/331: OpenIPMI-modalias-2.0.19-9.fc20.x86_64 >2014-08-10 14:06:34,497 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 242/331: OpenIPMI-modalias-2.0.19-9.fc20.x86_64. >2014-08-10 14:06:34,749 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 243/331: ipmitool-1.8.13-3.fc20.x86_64 >2014-08-10 14:06:34,761 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 243/331: ipmitool-1.8.13-3.fc20.x86_64. >2014-08-10 14:06:35,222 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 244/331: fence-agents-ipmilan-4.0.10-1.fc20.x86_64 >2014-08-10 14:06:35,300 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 244/331: fence-agents-ipmilan-4.0.10-1.fc20.x86_64. >2014-08-10 14:06:35,490 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 245/331: ed-1.10-1.fc20.x86_64 >2014-08-10 14:06:35,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 245/331: ed-1.10-1.fc20.x86_64. >2014-08-10 14:06:35,767 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 246/331: keyutils-1.5.9-1.fc20.x86_64 >2014-08-10 14:06:35,780 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 246/331: keyutils-1.5.9-1.fc20.x86_64. >2014-08-10 14:06:36,589 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 247/331: 1:nfs-utils-1.3.0-2.1.fc20.x86_64 >2014-08-10 14:06:36,727 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 247/331: 1:nfs-utils-1.3.0-2.1.fc20.x86_64. >2014-08-10 14:06:37,916 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 248/331: tree-1.6.0-11.fc20.x86_64 >2014-08-10 14:06:37,933 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 248/331: tree-1.6.0-11.fc20.x86_64. >2014-08-10 14:06:38,193 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 249/331: netpbm-10.61.02-8.fc20.x86_64 >2014-08-10 14:06:38,206 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 249/331: netpbm-10.61.02-8.fc20.x86_64. >2014-08-10 14:06:38,507 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 250/331: libreport-filesystem-2.2.3-1.fc20.x86_64 >2014-08-10 14:06:38,521 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 250/331: libreport-filesystem-2.2.3-1.fc20.x86_64. >2014-08-10 14:06:38,781 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 251/331: mdadm-3.3-4.fc20.x86_64 >2014-08-10 14:06:38,794 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 251/331: mdadm-3.3-4.fc20.x86_64. >2014-08-10 14:06:39,277 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 252/331: cyrus-sasl-md5-2.1.26-14.fc20.x86_64 >2014-08-10 14:06:39,341 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 252/331: cyrus-sasl-md5-2.1.26-14.fc20.x86_64. >2014-08-10 14:06:39,603 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 253/331: libvirt-client-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:06:39,622 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 253/331: libvirt-client-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:06:40,780 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 254/331: libvirt-daemon-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:06:40,815 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 254/331: libvirt-daemon-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:06:41,410 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 255/331: libvirt-daemon-driver-network-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:06:41,427 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 255/331: libvirt-daemon-driver-network-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:06:42,043 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 256/331: libvirt-daemon-driver-qemu-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:06:42,081 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 256/331: libvirt-daemon-driver-qemu-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:06:42,371 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 257/331: libvirt-daemon-driver-nwfilter-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:06:42,379 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 257/331: libvirt-daemon-driver-nwfilter-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:06:42,749 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 258/331: libvirt-python-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:06:42,760 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 258/331: libvirt-python-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:06:43,129 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 259/331: mom-0.4.1-1.fc20.noarch >2014-08-10 14:06:43,150 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 259/331: mom-0.4.1-1.fc20.noarch. >2014-08-10 14:06:43,673 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 260/331: libvirt-daemon-driver-storage-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:06:43,729 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 260/331: libvirt-daemon-driver-storage-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:06:44,011 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 261/331: libvirt-lock-sanlock-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:06:44,019 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 261/331: libvirt-lock-sanlock-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:06:44,321 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 262/331: libvirt-daemon-driver-interface-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:06:44,333 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 262/331: libvirt-daemon-driver-interface-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:06:44,599 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 263/331: libvirt-daemon-driver-nodedev-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:06:44,615 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 263/331: libvirt-daemon-driver-nodedev-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:06:44,893 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 264/331: libvirt-daemon-config-nwfilter-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:06:44,905 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 264/331: libvirt-daemon-config-nwfilter-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:06:45,178 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 265/331: libvirt-daemon-driver-secret-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:06:45,211 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 265/331: libvirt-daemon-driver-secret-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:06:45,454 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 266/331: libconfig-1.4.9-5.fc20.x86_64 >2014-08-10 14:06:45,484 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 266/331: libconfig-1.4.9-5.fc20.x86_64. >2014-08-10 14:06:45,760 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 267/331: fontpackages-filesystem-1.44-9.fc20.noarch >2014-08-10 14:06:45,791 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 267/331: fontpackages-filesystem-1.44-9.fc20.noarch. >2014-08-10 14:06:46,018 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 268/331: fontconfig-2.11.0-2.fc20.x86_64 >2014-08-10 14:06:46,039 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 268/331: fontconfig-2.11.0-2.fc20.x86_64. >2014-08-10 14:06:47,602 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 269/331: ghostscript-fonts-5.50-32.fc20.noarch >2014-08-10 14:06:47,646 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 269/331: ghostscript-fonts-5.50-32.fc20.noarch. >2014-08-10 14:06:48,086 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 270/331: urw-fonts-2.4-18.fc20.noarch >2014-08-10 14:06:48,101 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 270/331: urw-fonts-2.4-18.fc20.noarch. >2014-08-10 14:06:48,867 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 271/331: kernel-tools-libs-3.15.8-200.fc20.x86_64 >2014-08-10 14:06:48,896 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 271/331: kernel-tools-libs-3.15.8-200.fc20.x86_64. >2014-08-10 14:06:49,110 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 272/331: kernel-tools-3.15.8-200.fc20.x86_64 >2014-08-10 14:06:49,120 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 272/331: kernel-tools-3.15.8-200.fc20.x86_64. >2014-08-10 14:06:49,561 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 273/331: perl-Class-Inspector-1.28-5.fc20.noarch >2014-08-10 14:06:49,675 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 273/331: perl-Class-Inspector-1.28-5.fc20.noarch. >2014-08-10 14:06:49,910 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 274/331: perl-SOAP-Transport-TCP-0.715-11.fc20.noarch >2014-08-10 14:06:50,006 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 274/331: perl-SOAP-Transport-TCP-0.715-11.fc20.noarch. >2014-08-10 14:06:50,298 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 275/331: perl-SOAP-Lite-0.716-3.fc20.noarch >2014-08-10 14:06:50,312 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 275/331: perl-SOAP-Lite-0.716-3.fc20.noarch. >2014-08-10 14:06:51,032 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 276/331: libXau-1.0.8-2.fc20.x86_64 >2014-08-10 14:06:51,074 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 276/331: libXau-1.0.8-2.fc20.x86_64. >2014-08-10 14:06:51,334 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 277/331: libxcb-1.9.1-3.fc20.x86_64 >2014-08-10 14:06:51,347 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 277/331: libxcb-1.9.1-3.fc20.x86_64. >2014-08-10 14:06:52,260 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 278/331: libX11-1.6.1-1.fc20.x86_64 >2014-08-10 14:06:52,308 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 278/331: libX11-1.6.1-1.fc20.x86_64. >2014-08-10 14:06:52,630 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 279/331: libXext-1.3.2-2.fc20.x86_64 >2014-08-10 14:06:52,639 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 279/331: libXext-1.3.2-2.fc20.x86_64. >2014-08-10 14:06:52,932 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 280/331: libXrender-0.9.8-2.fc20.x86_64 >2014-08-10 14:06:52,945 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 280/331: libXrender-0.9.8-2.fc20.x86_64. >2014-08-10 14:06:53,262 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 281/331: gdk-pixbuf2-2.30.3-1.fc20.x86_64 >2014-08-10 14:06:53,301 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 281/331: gdk-pixbuf2-2.30.3-1.fc20.x86_64. >2014-08-10 14:06:54,001 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 282/331: libXfixes-5.0.1-2.fc20.x86_64 >2014-08-10 14:06:54,038 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 282/331: libXfixes-5.0.1-2.fc20.x86_64. >2014-08-10 14:06:54,364 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 283/331: libXdamage-1.1.4-4.fc20.x86_64 >2014-08-10 14:06:54,378 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 283/331: libXdamage-1.1.4-4.fc20.x86_64. >2014-08-10 14:06:54,700 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 284/331: libXi-1.7.4-1.fc20.x86_64 >2014-08-10 14:06:54,709 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 284/331: libXi-1.7.4-1.fc20.x86_64. >2014-08-10 14:06:55,043 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 285/331: libXtst-1.2.2-2.fc20.x86_64 >2014-08-10 14:06:55,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 285/331: libXtst-1.2.2-2.fc20.x86_64. >2014-08-10 14:06:55,454 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 286/331: brltty-4.5-9.fc20.x86_64 >2014-08-10 14:06:55,462 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 286/331: brltty-4.5-9.fc20.x86_64. >2014-08-10 14:06:56,447 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 287/331: brlapi-0.6.0-9.fc20.x86_64 >2014-08-10 14:06:56,503 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 287/331: brlapi-0.6.0-9.fc20.x86_64. >2014-08-10 14:06:57,075 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 288/331: libXcursor-1.1.14-2.fc20.x86_64 >2014-08-10 14:06:57,127 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 288/331: libXcursor-1.1.14-2.fc20.x86_64. >2014-08-10 14:06:57,497 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 289/331: libXrandr-1.4.1-2.fc20.x86_64 >2014-08-10 14:06:57,516 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 289/331: libXrandr-1.4.1-2.fc20.x86_64. >2014-08-10 14:06:57,820 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 290/331: libXinerama-1.1.3-2.fc20.x86_64 >2014-08-10 14:06:57,831 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 290/331: libXinerama-1.1.3-2.fc20.x86_64. >2014-08-10 14:06:58,163 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 291/331: libXcomposite-0.4.4-4.fc20.x86_64 >2014-08-10 14:06:58,170 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 291/331: libXcomposite-0.4.4-4.fc20.x86_64. >2014-08-10 14:06:58,490 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 292/331: mesa-libEGL-10.1.5-1.20140607.fc20.x86_64 >2014-08-10 14:06:58,502 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 292/331: mesa-libEGL-10.1.5-1.20140607.fc20.x86_64. >2014-08-10 14:06:58,875 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 293/331: libXft-2.3.1-5.fc20.x86_64 >2014-08-10 14:06:58,882 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 293/331: libXft-2.3.1-5.fc20.x86_64. >2014-08-10 14:06:59,210 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 294/331: libXxf86vm-1.1.3-2.fc20.x86_64 >2014-08-10 14:06:59,222 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 294/331: libXxf86vm-1.1.3-2.fc20.x86_64. >2014-08-10 14:06:59,487 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 295/331: mesa-libGL-10.1.5-1.20140607.fc20.x86_64 >2014-08-10 14:06:59,495 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 295/331: mesa-libGL-10.1.5-1.20140607.fc20.x86_64. >2014-08-10 14:06:59,845 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 296/331: cairo-1.13.1-0.1.git337ab1f.fc20.x86_64 >2014-08-10 14:06:59,859 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 296/331: cairo-1.13.1-0.1.git337ab1f.fc20.x86_64. >2014-08-10 14:07:00,286 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 297/331: pango-1.36.1-3.fc20.x86_64 >2014-08-10 14:07:00,306 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 297/331: pango-1.36.1-3.fc20.x86_64. >2014-08-10 14:07:00,705 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 298/331: cairo-gobject-1.13.1-0.1.git337ab1f.fc20.x86_64 >2014-08-10 14:07:00,844 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 298/331: cairo-gobject-1.13.1-0.1.git337ab1f.fc20.x86_64. >2014-08-10 14:07:01,000 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 299/331: gtk2-2.24.22-2.fc20.x86_64 >2014-08-10 14:07:01,010 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 299/331: gtk2-2.24.22-2.fc20.x86_64. >2014-08-10 14:07:01,892 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 300/331: vte-0.28.2-13.fc20.x86_64 >2014-08-10 14:07:01,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 300/331: vte-0.28.2-13.fc20.x86_64. >2014-08-10 14:07:03,048 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 301/331: amtterm-1.3-7.fc20.x86_64 >2014-08-10 14:07:03,063 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 301/331: amtterm-1.3-7.fc20.x86_64. >2014-08-10 14:07:03,532 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 302/331: fence-agents-amt-4.0.10-1.fc20.x86_64 >2014-08-10 14:07:03,610 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 302/331: fence-agents-amt-4.0.10-1.fc20.x86_64. >2014-08-10 14:07:03,830 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 303/331: fence-agents-all-4.0.10-1.fc20.x86_64 >2014-08-10 14:07:03,841 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 303/331: fence-agents-all-4.0.10-1.fc20.x86_64. >2014-08-10 14:07:04,176 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 304/331: libXevie-1.0.3-7.fc20.x86_64 >2014-08-10 14:07:04,189 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 304/331: libXevie-1.0.3-7.fc20.x86_64. >2014-08-10 14:07:04,512 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 305/331: at-spi2-core-2.10.2-1.fc20.x86_64 >2014-08-10 14:07:04,520 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 305/331: at-spi2-core-2.10.2-1.fc20.x86_64. >2014-08-10 14:07:05,030 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 306/331: at-spi2-atk-2.10.2-1.fc20.x86_64 >2014-08-10 14:07:05,067 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 306/331: at-spi2-atk-2.10.2-1.fc20.x86_64. >2014-08-10 14:07:05,315 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 307/331: gtk3-3.10.9-1.fc20.x86_64 >2014-08-10 14:07:05,357 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 307/331: gtk3-3.10.9-1.fc20.x86_64. >2014-08-10 14:07:06,604 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 308/331: vte3-0.34.9-3.fc20.x86_64 >2014-08-10 14:07:06,615 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 308/331: vte3-0.34.9-3.fc20.x86_64. >2014-08-10 14:07:07,052 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 309/331: libXt-1.1.4-7.fc20.x86_64 >2014-08-10 14:07:07,062 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 309/331: libXt-1.1.4-7.fc20.x86_64. >2014-08-10 14:07:07,376 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 310/331: ghostscript-9.14-3.fc20.x86_64 >2014-08-10 14:07:07,385 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 310/331: ghostscript-9.14-3.fc20.x86_64. >2014-08-10 14:07:08,461 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 311/331: netpbm-progs-10.61.02-8.fc20.x86_64 >2014-08-10 14:07:08,487 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 311/331: netpbm-progs-10.61.02-8.fc20.x86_64. >2014-08-10 14:07:09,703 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 312/331: gdisk-0.8.10-2.fc20.x86_64 >2014-08-10 14:07:09,746 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 312/331: gdisk-0.8.10-2.fc20.x86_64. >2014-08-10 14:07:10,040 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 313/331: vdsm-yajsonrpc-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:07:10,052 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 313/331: vdsm-yajsonrpc-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:07:10,361 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 314/331: vdsm-jsonrpc-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:07:10,400 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 314/331: vdsm-jsonrpc-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:07:10,705 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 315/331: binutils-2.23.88.0.1-13.fc20.x86_64 >2014-08-10 14:07:10,714 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 315/331: binutils-2.23.88.0.1-13.fc20.x86_64. >2014-08-10 14:07:12,028 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 316/331: bridge-utils-1.5-8.fc20.x86_64 >2014-08-10 14:07:12,065 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 316/331: bridge-utils-1.5-8.fc20.x86_64. >2014-08-10 14:07:12,304 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 317/331: libfdt-1.4.0-2.fc20.x86_64 >2014-08-10 14:07:12,313 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 317/331: libfdt-1.4.0-2.fc20.x86_64. >2014-08-10 14:07:13,039 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 318/331: libasyncns-0.8-6.fc20.x86_64 >2014-08-10 14:07:13,075 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 318/331: libasyncns-0.8-6.fc20.x86_64. >2014-08-10 14:07:13,402 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 319/331: pulseaudio-libs-5.0-7.fc20.x86_64 >2014-08-10 14:07:13,414 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 319/331: pulseaudio-libs-5.0-7.fc20.x86_64. >2014-08-10 14:07:14,009 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 320/331: 2:qemu-system-x86-1.6.2-7.fc20.x86_64 >2014-08-10 14:07:14,035 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 320/331: 2:qemu-system-x86-1.6.2-7.fc20.x86_64. >2014-08-10 14:07:15,010 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 321/331: 2:qemu-kvm-1.6.2-7.fc20.x86_64 >2014-08-10 14:07:15,029 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 321/331: 2:qemu-kvm-1.6.2-7.fc20.x86_64. >2014-08-10 14:07:15,519 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 322/331: libvirt-daemon-kvm-1.1.3.5-2.fc20.x86_64 >2014-08-10 14:07:15,601 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 322/331: libvirt-daemon-kvm-1.1.3.5-2.fc20.x86_64. >2014-08-10 14:07:15,998 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 323/331: 1:libguestfs-1.26.7-2.fc20.x86_64 >2014-08-10 14:07:16,014 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 323/331: 1:libguestfs-1.26.7-2.fc20.x86_64. >2014-08-10 14:07:16,986 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 324/331: 1:libguestfs-tools-c-1.26.7-2.fc20.x86_64 >2014-08-10 14:07:17,016 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 324/331: 1:libguestfs-tools-c-1.26.7-2.fc20.x86_64. >2014-08-10 14:07:18,247 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 325/331: audit-libs-python-2.3.7-1.fc20.x86_64 >2014-08-10 14:07:18,283 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 325/331: audit-libs-python-2.3.7-1.fc20.x86_64. >2014-08-10 14:07:18,569 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 326/331: policycoreutils-python-2.2.5-4.fc20.x86_64 >2014-08-10 14:07:18,581 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 326/331: policycoreutils-python-2.2.5-4.fc20.x86_64. >2014-08-10 14:07:19,772 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 327/331: python-netaddr-0.7.11-1.fc20.noarch >2014-08-10 14:07:19,846 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 327/331: python-netaddr-0.7.11-1.fc20.noarch. >2014-08-10 14:07:20,877 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 328/331: vdsm-4.16.1-6.gita4a4614.fc20.x86_64 >2014-08-10 14:07:20,900 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 328/331: vdsm-4.16.1-6.gita4a4614.fc20.x86_64. >2014-08-10 14:07:35,694 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 329/331: tuned-2.3.0-3.fc20.noarch >2014-08-10 14:07:35,720 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 329/331: tuned-2.3.0-3.fc20.noarch. >2014-08-10 14:07:36,537 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 330/331: vdsm-cli-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:07:36,556 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 330/331: vdsm-cli-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:07:36,878 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum install: 331/331: 2:qemu-kvm-tools-1.6.2-7.fc20.x86_64 >2014-08-10 14:07:36,887 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 331/331: 2:qemu-kvm-tools-1.6.2-7.fc20.x86_64. >2014-08-10 14:07:37,554 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 1/331: usbredir.x86_64 0:0.6-5.fc20 - u >2014-08-10 14:07:37,574 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 1/331: usbredir.x86_64 0:0.6-5.fc20 - u. >2014-08-10 14:07:37,706 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 2/331: gnupg.x86_64 0:1.4.18-1.fc20 - u >2014-08-10 14:07:37,715 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 2/331: gnupg.x86_64 0:1.4.18-1.fc20 - u. >2014-08-10 14:07:37,814 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 3/331: ioprocess.x86_64 0:0.5.0-1.fc20 - u >2014-08-10 14:07:37,839 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 3/331: ioprocess.x86_64 0:0.5.0-1.fc20 - u. >2014-08-10 14:07:37,956 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 4/331: python-netaddr.noarch 0:0.7.11-1.fc20 - u >2014-08-10 14:07:37,963 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 4/331: python-netaddr.noarch 0:0.7.11-1.fc20 - u. >2014-08-10 14:07:38,079 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 5/331: audit-libs-python.x86_64 0:2.3.7-1.fc20 - u >2014-08-10 14:07:38,088 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 5/331: audit-libs-python.x86_64 0:2.3.7-1.fc20 - u. >2014-08-10 14:07:38,221 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 6/331: fence-agents-amt.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:38,228 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 6/331: fence-agents-amt.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:38,369 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 7/331: dosfstools.x86_64 0:3.0.26-1.fc20 - u >2014-08-10 14:07:38,377 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 7/331: dosfstools.x86_64 0:3.0.26-1.fc20 - u. >2014-08-10 14:07:38,468 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 8/331: libasyncns.x86_64 0:0.8-6.fc20 - u >2014-08-10 14:07:38,477 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 8/331: libasyncns.x86_64 0:0.8-6.fc20 - u. >2014-08-10 14:07:38,886 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 9/331: pixman.x86_64 0:0.30.0-4.fc20 - u >2014-08-10 14:07:38,899 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 9/331: pixman.x86_64 0:0.30.0-4.fc20 - u. >2014-08-10 14:07:39,432 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 10/331: libfdt.x86_64 0:1.4.0-2.fc20 - u >2014-08-10 14:07:39,479 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 10/331: libfdt.x86_64 0:1.4.0-2.fc20 - u. >2014-08-10 14:07:39,530 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 11/331: perl-libwww-perl.noarch 0:6.05-3.fc20 - u >2014-08-10 14:07:39,545 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 11/331: perl-libwww-perl.noarch 0:6.05-3.fc20 - u. >2014-08-10 14:07:39,630 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 12/331: perl-HTTP-Date.noarch 0:6.02-8.fc20 - u >2014-08-10 14:07:39,636 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 12/331: perl-HTTP-Date.noarch 0:6.02-8.fc20 - u. >2014-08-10 14:07:39,753 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 13/331: python-pyudev.noarch 0:0.15-5.fc20 - u >2014-08-10 14:07:39,760 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 13/331: python-pyudev.noarch 0:0.15-5.fc20 - u. >2014-08-10 14:07:39,852 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 14/331: bridge-utils.x86_64 0:1.5-8.fc20 - u >2014-08-10 14:07:39,859 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 14/331: bridge-utils.x86_64 0:1.5-8.fc20 - u. >2014-08-10 14:07:39,993 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 15/331: libgusb.x86_64 0:0.1.6-2.fc20 - u >2014-08-10 14:07:40,000 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 15/331: libgusb.x86_64 0:0.1.6-2.fc20 - u. >2014-08-10 14:07:40,100 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 16/331: libXxf86vm.x86_64 0:1.1.3-2.fc20 - u >2014-08-10 14:07:40,108 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 16/331: libXxf86vm.x86_64 0:1.1.3-2.fc20 - u. >2014-08-10 14:07:40,202 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 17/331: binutils.x86_64 0:2.23.88.0.1-13.fc20 - u >2014-08-10 14:07:40,216 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 17/331: binutils.x86_64 0:2.23.88.0.1-13.fc20 - u. >2014-08-10 14:07:40,333 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 18/331: syslinux-extlinux.x86_64 0:4.05-7.fc20 - u >2014-08-10 14:07:40,340 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 18/331: syslinux-extlinux.x86_64 0:4.05-7.fc20 - u. >2014-08-10 14:07:40,449 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 19/331: lzo.x86_64 0:2.08-1.fc20 - u >2014-08-10 14:07:40,456 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 19/331: lzo.x86_64 0:2.08-1.fc20 - u. >2014-08-10 14:07:40,549 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 20/331: libxslt.x86_64 0:1.1.28-5.fc20 - u >2014-08-10 14:07:40,555 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 20/331: libxslt.x86_64 0:1.1.28-5.fc20 - u. >2014-08-10 14:07:40,672 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 21/331: vdsm-yajsonrpc.noarch 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:07:40,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 21/331: vdsm-yajsonrpc.noarch 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:07:40,770 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 22/331: qemu-img.x86_64 2:1.6.2-7.fc20 - u >2014-08-10 14:07:40,793 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 22/331: qemu-img.x86_64 2:1.6.2-7.fc20 - u. >2014-08-10 14:07:40,879 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 23/331: lcms2.x86_64 0:2.6-1.fc20 - u >2014-08-10 14:07:40,895 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 23/331: lcms2.x86_64 0:2.6-1.fc20 - u. >2014-08-10 14:07:41,002 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 24/331: libXext.x86_64 0:1.3.2-2.fc20 - u >2014-08-10 14:07:41,010 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 24/331: libXext.x86_64 0:1.3.2-2.fc20 - u. >2014-08-10 14:07:41,101 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 25/331: libvirt-daemon-kvm.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:07:41,110 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 25/331: libvirt-daemon-kvm.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:07:41,234 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 26/331: glusterfs.x86_64 0:3.5.1-1.fc20 - u >2014-08-10 14:07:41,242 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 26/331: glusterfs.x86_64 0:3.5.1-1.fc20 - u. >2014-08-10 14:07:41,332 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 27/331: mesa-libgbm.x86_64 0:10.1.5-1.20140607.fc20 - u >2014-08-10 14:07:41,342 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 27/331: mesa-libgbm.x86_64 0:10.1.5-1.20140607.fc20 - u. >2014-08-10 14:07:41,432 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 28/331: libguestfs.x86_64 1:1.26.7-2.fc20 - u >2014-08-10 14:07:41,441 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 28/331: libguestfs.x86_64 1:1.26.7-2.fc20 - u. >2014-08-10 14:07:41,539 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 29/331: gdisk.x86_64 0:0.8.10-2.fc20 - u >2014-08-10 14:07:41,549 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 29/331: gdisk.x86_64 0:0.8.10-2.fc20 - u. >2014-08-10 14:07:41,637 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 30/331: qemu-system-x86.x86_64 2:1.6.2-7.fc20 - u >2014-08-10 14:07:41,648 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 30/331: qemu-system-x86.x86_64 2:1.6.2-7.fc20 - u. >2014-08-10 14:07:41,745 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 31/331: device-mapper-multipath.x86_64 0:0.4.9-56.fc20 - u >2014-08-10 14:07:41,756 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 31/331: device-mapper-multipath.x86_64 0:0.4.9-56.fc20 - u. >2014-08-10 14:07:41,871 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 32/331: libXau.x86_64 0:1.0.8-2.fc20 - u >2014-08-10 14:07:41,880 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 32/331: libXau.x86_64 0:1.0.8-2.fc20 - u. >2014-08-10 14:07:41,969 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 33/331: sanlock.x86_64 0:3.0.0-2.fc20 - u >2014-08-10 14:07:41,979 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 33/331: sanlock.x86_64 0:3.0.0-2.fc20 - u. >2014-08-10 14:07:42,094 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 34/331: lzop.x86_64 0:1.03-9.fc20 - u >2014-08-10 14:07:42,103 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 34/331: lzop.x86_64 0:1.03-9.fc20 - u. >2014-08-10 14:07:42,230 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 35/331: bzip2.x86_64 0:1.0.6-9.fc20 - u >2014-08-10 14:07:42,244 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 35/331: bzip2.x86_64 0:1.0.6-9.fc20 - u. >2014-08-10 14:07:42,362 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 36/331: libXi.x86_64 0:1.7.4-1.fc20 - u >2014-08-10 14:07:42,377 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 36/331: libXi.x86_64 0:1.7.4-1.fc20 - u. >2014-08-10 14:07:42,466 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 37/331: fence-agents-rhevm.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:42,476 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 37/331: fence-agents-rhevm.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:42,564 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 38/331: libwayland-cursor.x86_64 0:1.2.0-3.fc20 - u >2014-08-10 14:07:42,575 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 38/331: libwayland-cursor.x86_64 0:1.2.0-3.fc20 - u. >2014-08-10 14:07:42,664 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 39/331: python-suds.noarch 0:0.4.1-5.fc20 - u >2014-08-10 14:07:42,675 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 39/331: python-suds.noarch 0:0.4.1-5.fc20 - u. >2014-08-10 14:07:42,787 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 40/331: libvirt-daemon-driver-storage.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:07:42,799 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 40/331: libvirt-daemon-driver-storage.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:07:42,894 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 41/331: libevent.x86_64 0:2.0.21-3.fc20 - u >2014-08-10 14:07:42,931 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 41/331: libevent.x86_64 0:2.0.21-3.fc20 - u. >2014-08-10 14:07:43,002 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 42/331: perl-Class-Inspector.noarch 0:1.28-5.fc20 - u >2014-08-10 14:07:43,014 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 42/331: perl-Class-Inspector.noarch 0:1.28-5.fc20 - u. >2014-08-10 14:07:43,126 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 43/331: netpbm-progs.x86_64 0:10.61.02-8.fc20 - u >2014-08-10 14:07:43,138 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 43/331: netpbm-progs.x86_64 0:10.61.02-8.fc20 - u. >2014-08-10 14:07:43,249 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 44/331: fence-agents-apc.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:43,262 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 44/331: fence-agents-apc.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:43,382 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 45/331: kernel-tools-libs.x86_64 0:3.15.8-200.fc20 - u >2014-08-10 14:07:43,395 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 45/331: kernel-tools-libs.x86_64 0:3.15.8-200.fc20 - u. >2014-08-10 14:07:43,491 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 46/331: libguestfs-tools-c.x86_64 1:1.26.7-2.fc20 - u >2014-08-10 14:07:43,503 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 46/331: libguestfs-tools-c.x86_64 1:1.26.7-2.fc20 - u. >2014-08-10 14:07:43,652 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 47/331: fence-agents-kdump.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:43,660 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 47/331: fence-agents-kdump.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:44,129 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 48/331: gtk3.x86_64 0:3.10.9-1.fc20 - u >2014-08-10 14:07:44,198 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 48/331: gtk3.x86_64 0:3.10.9-1.fc20 - u. >2014-08-10 14:07:44,301 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 49/331: libXcomposite.x86_64 0:0.4.4-4.fc20 - u >2014-08-10 14:07:44,314 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 49/331: libXcomposite.x86_64 0:0.4.4-4.fc20 - u. >2014-08-10 14:07:44,428 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 50/331: glusterfs-api.x86_64 0:3.5.1-1.fc20 - u >2014-08-10 14:07:44,438 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 50/331: glusterfs-api.x86_64 0:3.5.1-1.fc20 - u. >2014-08-10 14:07:44,529 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 51/331: fontpackages-filesystem.noarch 0:1.44-9.fc20 - u >2014-08-10 14:07:44,538 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 51/331: fontpackages-filesystem.noarch 0:1.44-9.fc20 - u. >2014-08-10 14:07:44,631 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 52/331: perl-SOAP-Lite.noarch 0:0.716-3.fc20 - u >2014-08-10 14:07:44,645 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 52/331: perl-SOAP-Lite.noarch 0:0.716-3.fc20 - u. >2014-08-10 14:07:44,733 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 53/331: libconfig.x86_64 0:1.4.9-5.fc20 - u >2014-08-10 14:07:44,745 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 53/331: libconfig.x86_64 0:1.4.9-5.fc20 - u. >2014-08-10 14:07:44,867 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 54/331: libvirt-lock-sanlock.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:07:44,877 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 54/331: libvirt-lock-sanlock.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:07:44,969 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 55/331: ntfs-3g.x86_64 2:2014.2.15-1.fc20 - u >2014-08-10 14:07:44,976 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 55/331: ntfs-3g.x86_64 2:2014.2.15-1.fc20 - u. >2014-08-10 14:07:45,096 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 56/331: fence-agents-alom.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:45,109 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 56/331: fence-agents-alom.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:45,197 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 57/331: cyrus-sasl-md5.x86_64 0:2.1.26-14.fc20 - u >2014-08-10 14:07:45,208 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 57/331: cyrus-sasl-md5.x86_64 0:2.1.26-14.fc20 - u. >2014-08-10 14:07:45,307 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 58/331: sos.noarch 0:3.1-1.1.fc20.ovirt - u >2014-08-10 14:07:45,316 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 58/331: sos.noarch 0:3.1-1.1.fc20.ovirt - u. >2014-08-10 14:07:45,411 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 59/331: libXrandr.x86_64 0:1.4.1-2.fc20 - u >2014-08-10 14:07:45,423 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 59/331: libXrandr.x86_64 0:1.4.1-2.fc20 - u. >2014-08-10 14:07:45,545 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 60/331: fence-agents-lpar.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:45,556 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 60/331: fence-agents-lpar.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:45,647 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 61/331: vdsm-xmlrpc.noarch 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:07:45,655 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 61/331: vdsm-xmlrpc.noarch 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:07:45,761 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 62/331: fence-agents-wti.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:45,771 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 62/331: fence-agents-wti.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:45,866 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 63/331: perl-Data-Dumper.x86_64 0:2.151-1.fc20 - u >2014-08-10 14:07:45,879 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 63/331: perl-Data-Dumper.x86_64 0:2.151-1.fc20 - u. >2014-08-10 14:07:45,968 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 64/331: ntfsprogs.x86_64 2:2014.2.15-1.fc20 - u >2014-08-10 14:07:45,993 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 64/331: ntfsprogs.x86_64 2:2014.2.15-1.fc20 - u. >2014-08-10 14:07:46,069 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 65/331: sg3_utils.x86_64 0:1.37-2.fc20 - u >2014-08-10 14:07:46,086 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 65/331: sg3_utils.x86_64 0:1.37-2.fc20 - u. >2014-08-10 14:07:46,196 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 66/331: perl-HTML-Parser.x86_64 0:3.71-4.fc20 - u >2014-08-10 14:07:46,210 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 66/331: perl-HTML-Parser.x86_64 0:3.71-4.fc20 - u. >2014-08-10 14:07:46,298 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 67/331: fence-agents-ovh.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:46,310 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 67/331: fence-agents-ovh.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:46,399 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 68/331: libreport-filesystem.x86_64 0:2.2.3-1.fc20 - u >2014-08-10 14:07:46,409 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 68/331: libreport-filesystem.x86_64 0:2.2.3-1.fc20 - u. >2014-08-10 14:07:46,501 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 69/331: mdadm.x86_64 0:3.3-4.fc20 - u >2014-08-10 14:07:46,517 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 69/331: mdadm.x86_64 0:3.3-4.fc20 - u. >2014-08-10 14:07:46,603 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 70/331: glusterfs-cli.x86_64 0:3.5.1-1.fc20 - u >2014-08-10 14:07:46,616 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 70/331: glusterfs-cli.x86_64 0:3.5.1-1.fc20 - u. >2014-08-10 14:07:46,730 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 71/331: perl-Encode-Locale.noarch 0:1.03-7.fc20 - u >2014-08-10 14:07:46,740 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 71/331: perl-Encode-Locale.noarch 0:1.03-7.fc20 - u. >2014-08-10 14:07:46,858 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 72/331: boost-thread.x86_64 0:1.54.0-9.fc20 - u >2014-08-10 14:07:46,864 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 72/331: boost-thread.x86_64 0:1.54.0-9.fc20 - u. >2014-08-10 14:07:46,993 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 73/331: pulseaudio-libs.x86_64 0:5.0-7.fc20 - u >2014-08-10 14:07:47,005 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 73/331: pulseaudio-libs.x86_64 0:5.0-7.fc20 - u. >2014-08-10 14:07:47,098 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 74/331: netpbm.x86_64 0:10.61.02-8.fc20 - u >2014-08-10 14:07:47,104 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 74/331: netpbm.x86_64 0:10.61.02-8.fc20 - u. >2014-08-10 14:07:47,203 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 75/331: libusbx.x86_64 0:1.0.19-1.fc20 - u >2014-08-10 14:07:47,212 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 75/331: libusbx.x86_64 0:1.0.19-1.fc20 - u. >2014-08-10 14:07:47,306 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 76/331: tree.x86_64 0:1.6.0-11.fc20 - u >2014-08-10 14:07:47,320 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 76/331: tree.x86_64 0:1.6.0-11.fc20 - u. >2014-08-10 14:07:47,440 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 77/331: perl-MIME-tools.noarch 0:5.505-1.fc20 - u >2014-08-10 14:07:47,452 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 77/331: perl-MIME-tools.noarch 0:5.505-1.fc20 - u. >2014-08-10 14:07:47,567 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 78/331: leveldb.x86_64 0:1.12.0-5.fc20 - u >2014-08-10 14:07:47,576 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 78/331: leveldb.x86_64 0:1.12.0-5.fc20 - u. >2014-08-10 14:07:47,692 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 79/331: ethtool.x86_64 2:3.10-2.fc20 - u >2014-08-10 14:07:47,701 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 79/331: ethtool.x86_64 2:3.10-2.fc20 - u. >2014-08-10 14:07:47,820 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 80/331: fence-agents-netio.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:47,833 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 80/331: fence-agents-netio.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:47,922 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 81/331: vte.x86_64 0:0.28.2-13.fc20 - u >2014-08-10 14:07:47,966 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 81/331: vte.x86_64 0:0.28.2-13.fc20 - u. >2014-08-10 14:07:48,047 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 82/331: vdsm.x86_64 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:07:48,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 82/331: vdsm.x86_64 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:07:48,151 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 83/331: python-backports-ssl_match_hostname.noarch 0:3.4.0.2-1.fc20 - u >2014-08-10 14:07:48,164 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 83/331: python-backports-ssl_match_hostname.noarch 0:3.4.0.2-1.fc20 - u. >2014-08-10 14:07:48,253 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 84/331: fence-agents-ifmib.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:48,264 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 84/331: fence-agents-ifmib.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:48,354 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 85/331: at-spi2-atk.x86_64 0:2.10.2-1.fc20 - u >2014-08-10 14:07:48,363 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 85/331: at-spi2-atk.x86_64 0:2.10.2-1.fc20 - u. >2014-08-10 14:07:48,481 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 86/331: python-kitchen.noarch 0:1.1.1-5.fc20 - u >2014-08-10 14:07:48,487 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 86/331: python-kitchen.noarch 0:1.1.1-5.fc20 - u. >2014-08-10 14:07:48,586 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 87/331: telnet.x86_64 1:0.17-58.fc20 - u >2014-08-10 14:07:48,595 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 87/331: telnet.x86_64 1:0.17-58.fc20 - u. >2014-08-10 14:07:48,689 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 88/331: m2crypto.x86_64 0:0.21.1-13.fc20 - u >2014-08-10 14:07:48,703 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 88/331: m2crypto.x86_64 0:0.21.1-13.fc20 - u. >2014-08-10 14:07:49,455 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 89/331: snappy.x86_64 0:1.1.0-2.fc20 - u >2014-08-10 14:07:49,497 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 89/331: snappy.x86_64 0:1.1.0-2.fc20 - u. >2014-08-10 14:07:49,557 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 90/331: libwayland-client.x86_64 0:1.2.0-3.fc20 - u >2014-08-10 14:07:49,564 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 90/331: libwayland-client.x86_64 0:1.2.0-3.fc20 - u. >2014-08-10 14:07:49,659 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 91/331: gnutls-dane.x86_64 0:3.1.25-1.fc20 - u >2014-08-10 14:07:49,671 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 91/331: gnutls-dane.x86_64 0:3.1.25-1.fc20 - u. >2014-08-10 14:07:49,785 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 92/331: perl-WWW-RobotRules.noarch 0:6.02-8.fc20 - u >2014-08-10 14:07:49,795 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 92/331: perl-WWW-RobotRules.noarch 0:6.02-8.fc20 - u. >2014-08-10 14:07:49,912 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 93/331: perl-MIME-Lite.noarch 0:3.030-1.fc20 - u >2014-08-10 14:07:49,920 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 93/331: perl-MIME-Lite.noarch 0:3.030-1.fc20 - u. >2014-08-10 14:07:50,047 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 94/331: corosynclib.x86_64 0:2.3.3-2.fc20 - u >2014-08-10 14:07:50,069 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 94/331: corosynclib.x86_64 0:2.3.3-2.fc20 - u. >2014-08-10 14:07:50,150 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 95/331: ghostscript-fonts.noarch 0:5.50-32.fc20 - u >2014-08-10 14:07:50,160 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 95/331: ghostscript-fonts.noarch 0:5.50-32.fc20 - u. >2014-08-10 14:07:50,252 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 96/331: libSM.x86_64 0:1.2.1-6.fc20 - u >2014-08-10 14:07:50,259 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 96/331: libSM.x86_64 0:1.2.1-6.fc20 - u. >2014-08-10 14:07:50,378 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 97/331: ntp.x86_64 0:4.2.6p5-18.fc20 - u >2014-08-10 14:07:50,383 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 97/331: ntp.x86_64 0:4.2.6p5-18.fc20 - u. >2014-08-10 14:07:50,480 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 98/331: keyutils.x86_64 0:1.5.9-1.fc20 - u >2014-08-10 14:07:50,491 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 98/331: keyutils.x86_64 0:1.5.9-1.fc20 - u. >2014-08-10 14:07:50,582 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 99/331: ed.x86_64 0:1.10-1.fc20 - u >2014-08-10 14:07:50,590 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 99/331: ed.x86_64 0:1.10-1.fc20 - u. >2014-08-10 14:07:50,708 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 100/331: fence-agents-ipdu.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:50,715 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 100/331: fence-agents-ipdu.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:50,810 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 101/331: qemu-common.x86_64 2:1.6.2-7.fc20 - u >2014-08-10 14:07:50,822 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 101/331: qemu-common.x86_64 2:1.6.2-7.fc20 - u. >2014-08-10 14:07:50,920 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 102/331: augeas.x86_64 0:1.2.0-2.fc20 - u >2014-08-10 14:07:50,930 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 102/331: augeas.x86_64 0:1.2.0-2.fc20 - u. >2014-08-10 14:07:51,021 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 103/331: perl-Business-ISBN.noarch 0:2.06-4.fc20 - u >2014-08-10 14:07:51,062 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 103/331: perl-Business-ISBN.noarch 0:2.06-4.fc20 - u. >2014-08-10 14:07:51,148 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 104/331: cairo.x86_64 0:1.13.1-0.1.git337ab1f.fc20 - u >2014-08-10 14:07:51,162 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 104/331: cairo.x86_64 0:1.13.1-0.1.git337ab1f.fc20 - u. >2014-08-10 14:07:51,275 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 105/331: rpcbind.x86_64 0:0.2.1-0.2.fc20 - u >2014-08-10 14:07:51,286 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 105/331: rpcbind.x86_64 0:0.2.1-0.2.fc20 - u. >2014-08-10 14:07:51,394 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 106/331: OpenIPMI-modalias.x86_64 0:2.0.19-9.fc20 - u >2014-08-10 14:07:51,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 106/331: OpenIPMI-modalias.x86_64 0:2.0.19-9.fc20 - u. >2014-08-10 14:07:51,496 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 107/331: sanlock-python.x86_64 0:3.0.0-2.fc20 - u >2014-08-10 14:07:51,510 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 107/331: sanlock-python.x86_64 0:3.0.0-2.fc20 - u. >2014-08-10 14:07:51,597 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 108/331: libXt.x86_64 0:1.1.4-7.fc20 - u >2014-08-10 14:07:51,609 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 108/331: libXt.x86_64 0:1.1.4-7.fc20 - u. >2014-08-10 14:07:51,733 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 109/331: graphite2.x86_64 0:1.2.2-4.fc20 - u >2014-08-10 14:07:51,742 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 109/331: graphite2.x86_64 0:1.2.2-4.fc20 - u. >2014-08-10 14:07:51,860 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 110/331: perl-Net-SSLeay.x86_64 0:1.55-4.fc20 - u >2014-08-10 14:07:51,866 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 110/331: perl-Net-SSLeay.x86_64 0:1.55-4.fc20 - u. >2014-08-10 14:07:51,960 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 111/331: btrfs-progs.x86_64 0:3.14.2-3.fc20 - u >2014-08-10 14:07:51,973 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 111/331: btrfs-progs.x86_64 0:3.14.2-3.fc20 - u. >2014-08-10 14:07:52,062 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 112/331: json-c.x86_64 0:0.11-6.fc20 - u >2014-08-10 14:07:52,073 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 112/331: json-c.x86_64 0:0.11-6.fc20 - u. >2014-08-10 14:07:52,198 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 113/331: tuned.noarch 0:2.3.0-3.fc20 - u >2014-08-10 14:07:52,205 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 113/331: tuned.noarch 0:2.3.0-3.fc20 - u. >2014-08-10 14:07:52,298 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 114/331: cryptsetup.x86_64 0:1.6.4-1.fc20 - u >2014-08-10 14:07:52,304 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 114/331: cryptsetup.x86_64 0:1.6.4-1.fc20 - u. >2014-08-10 14:07:52,401 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 115/331: urw-fonts.noarch 0:2.4-18.fc20 - u >2014-08-10 14:07:52,412 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 115/331: urw-fonts.noarch 0:2.4-18.fc20 - u. >2014-08-10 14:07:52,528 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 116/331: python-cpopen.x86_64 0:1.3-4.fc20 - u >2014-08-10 14:07:52,536 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 116/331: python-cpopen.x86_64 0:1.3-4.fc20 - u. >2014-08-10 14:07:52,628 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 117/331: perl-HTTP-Message.noarch 0:6.06-7.fc20 - u >2014-08-10 14:07:52,636 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 117/331: perl-HTTP-Message.noarch 0:6.06-7.fc20 - u. >2014-08-10 14:07:52,725 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 118/331: ntpdate.x86_64 0:4.2.6p5-18.fc20 - u >2014-08-10 14:07:52,735 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 118/331: ntpdate.x86_64 0:4.2.6p5-18.fc20 - u. >2014-08-10 14:07:52,825 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 119/331: sheepdog.x86_64 0:0.3.0-5.fc20 - u >2014-08-10 14:07:52,834 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 119/331: sheepdog.x86_64 0:0.3.0-5.fc20 - u. >2014-08-10 14:07:52,952 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 120/331: libXfixes.x86_64 0:5.0.1-2.fc20 - u >2014-08-10 14:07:52,967 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 120/331: libXfixes.x86_64 0:5.0.1-2.fc20 - u. >2014-08-10 14:07:53,054 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 121/331: libxkbcommon.x86_64 0:0.3.1-1.fc20 - u >2014-08-10 14:07:53,066 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 121/331: libxkbcommon.x86_64 0:0.3.1-1.fc20 - u. >2014-08-10 14:07:53,155 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 122/331: fence-agents-eaton-snmp.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:53,166 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 122/331: fence-agents-eaton-snmp.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:53,282 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 123/331: libxcb.x86_64 0:1.9.1-3.fc20 - u >2014-08-10 14:07:53,290 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 123/331: libxcb.x86_64 0:1.9.1-3.fc20 - u. >2014-08-10 14:07:53,384 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 124/331: mom.noarch 0:0.4.1-1.fc20 - u >2014-08-10 14:07:53,397 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 124/331: mom.noarch 0:0.4.1-1.fc20 - u. >2014-08-10 14:07:53,484 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 125/331: augeas-libs.x86_64 0:1.2.0-2.fc20 - u >2014-08-10 14:07:53,497 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 125/331: augeas-libs.x86_64 0:1.2.0-2.fc20 - u. >2014-08-10 14:07:53,587 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 126/331: psmisc.x86_64 0:22.20-3.fc20 - u >2014-08-10 14:07:53,596 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 126/331: psmisc.x86_64 0:22.20-3.fc20 - u. >2014-08-10 14:07:54,092 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 127/331: vte3.x86_64 0:0.34.9-3.fc20 - u >2014-08-10 14:07:54,151 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 127/331: vte3.x86_64 0:0.34.9-3.fc20 - u. >2014-08-10 14:07:54,214 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 128/331: python-chardet.noarch 0:2.0.1-7.fc20 - u >2014-08-10 14:07:54,225 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 128/331: python-chardet.noarch 0:2.0.1-7.fc20 - u. >2014-08-10 14:07:54,316 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 129/331: kernel-tools.x86_64 0:3.15.8-200.fc20 - u >2014-08-10 14:07:54,325 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 129/331: kernel-tools.x86_64 0:3.15.8-200.fc20 - u. >2014-08-10 14:07:54,452 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 130/331: libXfont.x86_64 0:1.4.8-1.fc20 - u >2014-08-10 14:07:54,466 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 130/331: libXfont.x86_64 0:1.4.8-1.fc20 - u. >2014-08-10 14:07:54,587 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 131/331: autogen-libopts.x86_64 0:5.18-1.fc20 - u >2014-08-10 14:07:54,598 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 131/331: autogen-libopts.x86_64 0:5.18-1.fc20 - u. >2014-08-10 14:07:54,689 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 132/331: fence-agents-all.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:54,697 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 132/331: fence-agents-all.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:54,790 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 133/331: libibverbs.x86_64 0:1.1.7-3.fc20 - u >2014-08-10 14:07:54,797 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 133/331: libibverbs.x86_64 0:1.1.7-3.fc20 - u. >2014-08-10 14:07:54,892 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 134/331: perl-Business-ISBN-Data.noarch 0:20120719.001-4.fc20 - u >2014-08-10 14:07:54,904 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 134/331: perl-Business-ISBN-Data.noarch 0:20120719.001-4.fc20 - u. >2014-08-10 14:07:55,027 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 135/331: fence-agents-eps.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:55,037 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 135/331: fence-agents-eps.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:55,153 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 136/331: fence-agents-hpblade.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:55,161 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 136/331: fence-agents-hpblade.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:55,255 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 137/331: jbigkit-libs.x86_64 0:2.0-10.fc20 - u >2014-08-10 14:07:55,269 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 137/331: jbigkit-libs.x86_64 0:2.0-10.fc20 - u. >2014-08-10 14:07:55,391 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 138/331: cyrus-sasl.x86_64 0:2.1.26-14.fc20 - u >2014-08-10 14:07:55,401 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 138/331: cyrus-sasl.x86_64 0:2.1.26-14.fc20 - u. >2014-08-10 14:07:55,500 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 139/331: libtiff.x86_64 0:4.0.3-15.fc20 - u >2014-08-10 14:07:55,509 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 139/331: libtiff.x86_64 0:4.0.3-15.fc20 - u. >2014-08-10 14:07:55,602 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 140/331: supermin.x86_64 0:5.1.9-1.fc20 - u >2014-08-10 14:07:55,608 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 140/331: supermin.x86_64 0:5.1.9-1.fc20 - u. >2014-08-10 14:07:55,703 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 141/331: sanlock-lib.x86_64 0:3.0.0-2.fc20 - u >2014-08-10 14:07:55,716 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 141/331: sanlock-lib.x86_64 0:3.0.0-2.fc20 - u. >2014-08-10 14:07:55,806 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 142/331: uml_utilities.x86_64 0:20070815-12.fc20 - u >2014-08-10 14:07:55,825 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 142/331: uml_utilities.x86_64 0:20070815-12.fc20 - u. >2014-08-10 14:07:55,933 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 143/331: libXinerama.x86_64 0:1.1.3-2.fc20 - u >2014-08-10 14:07:55,941 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 143/331: libXinerama.x86_64 0:1.1.3-2.fc20 - u. >2014-08-10 14:07:56,068 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 144/331: perl-IO-Compress.noarch 0:2.062-2.fc20 - u >2014-08-10 14:07:56,094 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 144/331: perl-IO-Compress.noarch 0:2.062-2.fc20 - u. >2014-08-10 14:07:56,170 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 145/331: vdsm-python-zombiereaper.noarch 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:07:56,189 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 145/331: vdsm-python-zombiereaper.noarch 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:07:56,325 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 146/331: glusterfs-rdma.x86_64 0:3.5.1-1.fc20 - u >2014-08-10 14:07:56,338 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 146/331: glusterfs-rdma.x86_64 0:3.5.1-1.fc20 - u. >2014-08-10 14:07:56,427 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 147/331: perl-hivex.x86_64 0:1.3.8-3.fc20 - u >2014-08-10 14:07:56,438 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 147/331: perl-hivex.x86_64 0:1.3.8-3.fc20 - u. >2014-08-10 14:07:56,590 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 148/331: perl-HTTP-Daemon.noarch 0:6.01-7.fc20 - u >2014-08-10 14:07:56,603 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 148/331: perl-HTTP-Daemon.noarch 0:6.01-7.fc20 - u. >2014-08-10 14:07:56,726 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 149/331: scrub.x86_64 0:2.5.2-4.fc20 - u >2014-08-10 14:07:56,736 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 149/331: scrub.x86_64 0:2.5.2-4.fc20 - u. >2014-08-10 14:07:56,827 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 150/331: libvirt-daemon-driver-nwfilter.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:07:56,835 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 150/331: libvirt-daemon-driver-nwfilter.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:07:56,962 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 151/331: ipxe-roms-qemu.noarch 0:20140303-1.gitff1e7fc7.fc20 - u >2014-08-10 14:07:57,017 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 151/331: ipxe-roms-qemu.noarch 0:20140303-1.gitff1e7fc7.fc20 - u. >2014-08-10 14:07:57,089 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 152/331: libogg.x86_64 2:1.3.0-6.fc20 - u >2014-08-10 14:07:57,100 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 152/331: libogg.x86_64 2:1.3.0-6.fc20 - u. >2014-08-10 14:07:57,225 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 153/331: libldm.x86_64 0:0.2.3-4.fc20 - u >2014-08-10 14:07:57,241 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 153/331: libldm.x86_64 0:0.2.3-4.fc20 - u. >2014-08-10 14:07:57,327 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 154/331: libjpeg-turbo.x86_64 0:1.3.1-2.fc20 - u >2014-08-10 14:07:57,340 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 154/331: libjpeg-turbo.x86_64 0:1.3.1-2.fc20 - u. >2014-08-10 14:07:57,428 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 155/331: checkpolicy.x86_64 0:2.1.12-5.fc20 - u >2014-08-10 14:07:57,439 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 155/331: checkpolicy.x86_64 0:2.1.12-5.fc20 - u. >2014-08-10 14:07:57,530 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 156/331: policycoreutils-python.x86_64 0:2.2.5-4.fc20 - u >2014-08-10 14:07:57,539 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 156/331: policycoreutils-python.x86_64 0:2.2.5-4.fc20 - u. >2014-08-10 14:07:57,640 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 157/331: fence-agents-ipmilan.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:07:57,647 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 157/331: fence-agents-ipmilan.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:07:57,741 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 158/331: libX11-common.noarch 0:1.6.1-1.fc20 - u >2014-08-10 14:07:57,754 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 158/331: libX11-common.noarch 0:1.6.1-1.fc20 - u. >2014-08-10 14:07:57,876 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 159/331: gdk-pixbuf2.x86_64 0:2.30.3-1.fc20 - u >2014-08-10 14:07:57,887 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 159/331: gdk-pixbuf2.x86_64 0:2.30.3-1.fc20 - u. >2014-08-10 14:07:58,003 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 160/331: libwsman1.x86_64 0:2.4.3-1.fc20 - u >2014-08-10 14:07:58,027 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 160/331: libwsman1.x86_64 0:2.4.3-1.fc20 - u. >2014-08-10 14:07:58,109 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 161/331: librdmacm.x86_64 0:1.0.17-2.fc20 - u >2014-08-10 14:07:58,118 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 161/331: librdmacm.x86_64 0:1.0.17-2.fc20 - u. >2014-08-10 14:07:58,214 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 162/331: libvirt-daemon-driver-interface.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:07:58,226 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 162/331: libvirt-daemon-driver-interface.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:07:58,341 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 163/331: python-pthreading.noarch 0:0.1.3-3.fc20 - u >2014-08-10 14:07:58,350 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 163/331: python-pthreading.noarch 0:0.1.3-3.fc20 - u. >2014-08-10 14:07:58,492 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 164/331: json-glib.x86_64 0:0.16.2-1.fc20 - u >2014-08-10 14:07:58,500 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 164/331: json-glib.x86_64 0:0.16.2-1.fc20 - u. >2014-08-10 14:07:58,587 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 165/331: libcgroup.x86_64 0:0.38-8.fc20 - u >2014-08-10 14:07:58,599 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 165/331: libcgroup.x86_64 0:0.38-8.fc20 - u. >2014-08-10 14:07:58,689 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 166/331: spice-server.x86_64 0:0.12.5-2.fc20 - u >2014-08-10 14:07:58,698 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 166/331: spice-server.x86_64 0:0.12.5-2.fc20 - u. >2014-08-10 14:07:58,790 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 167/331: boost-system.x86_64 0:1.54.0-9.fc20 - u >2014-08-10 14:07:58,797 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 167/331: boost-system.x86_64 0:1.54.0-9.fc20 - u. >2014-08-10 14:07:58,924 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 168/331: perl-MIME-Types.noarch 0:2.04-1.fc20 - u >2014-08-10 14:07:58,938 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 168/331: perl-MIME-Types.noarch 0:2.04-1.fc20 - u. >2014-08-10 14:07:59,094 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 169/331: perl-IO-Socket-SSL.noarch 0:1.955-2.fc20 - u >2014-08-10 14:07:59,104 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 169/331: perl-IO-Socket-SSL.noarch 0:1.955-2.fc20 - u. >2014-08-10 14:07:59,221 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 170/331: perl-Net-HTTP.noarch 0:6.06-4.fc20 - u >2014-08-10 14:07:59,236 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 170/331: perl-Net-HTTP.noarch 0:6.06-4.fc20 - u. >2014-08-10 14:07:59,323 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 171/331: sgabios-bin.noarch 1:0.20110622svn-6.fc20 - u >2014-08-10 14:07:59,336 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 171/331: sgabios-bin.noarch 1:0.20110622svn-6.fc20 - u. >2014-08-10 14:07:59,453 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 172/331: fontconfig.x86_64 0:2.11.0-2.fc20 - u >2014-08-10 14:07:59,460 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 172/331: fontconfig.x86_64 0:2.11.0-2.fc20 - u. >2014-08-10 14:07:59,658 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 173/331: qemu-kvm.x86_64 2:1.6.2-7.fc20 - u >2014-08-10 14:07:59,667 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 173/331: qemu-kvm.x86_64 2:1.6.2-7.fc20 - u. >2014-08-10 14:07:59,794 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 174/331: brlapi.x86_64 0:0.6.0-9.fc20 - u >2014-08-10 14:07:59,799 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 174/331: brlapi.x86_64 0:0.6.0-9.fc20 - u. >2014-08-10 14:07:59,927 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 175/331: libqb.x86_64 0:0.17.0-1.fc20 - u >2014-08-10 14:07:59,940 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 175/331: libqb.x86_64 0:0.17.0-1.fc20 - u. >2014-08-10 14:08:00,030 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 176/331: libXtst.x86_64 0:1.2.2-2.fc20 - u >2014-08-10 14:08:00,081 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 176/331: libXtst.x86_64 0:1.2.2-2.fc20 - u. >2014-08-10 14:08:00,132 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 177/331: lsscsi.x86_64 0:0.27-2.fc20 - u >2014-08-10 14:08:00,139 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 177/331: lsscsi.x86_64 0:0.27-2.fc20 - u. >2014-08-10 14:08:00,232 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 178/331: fence-agents-cisco-ucs.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:00,263 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 178/331: fence-agents-cisco-ucs.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:00,359 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 179/331: setools-libs.x86_64 0:3.3.7-41.fc20 - u >2014-08-10 14:08:00,371 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 179/331: setools-libs.x86_64 0:3.3.7-41.fc20 - u. >2014-08-10 14:08:00,463 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 180/331: bluez-libs.x86_64 0:5.18-1.fc20 - u >2014-08-10 14:08:00,470 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 180/331: bluez-libs.x86_64 0:5.18-1.fc20 - u. >2014-08-10 14:08:00,564 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 181/331: ceph-libs.x86_64 0:0.81.0-5.fc20 - u >2014-08-10 14:08:00,578 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 181/331: ceph-libs.x86_64 0:0.81.0-5.fc20 - u. >2014-08-10 14:08:00,699 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 182/331: netcf-libs.x86_64 0:0.2.3-6.fc20 - u >2014-08-10 14:08:00,710 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 182/331: netcf-libs.x86_64 0:0.2.3-6.fc20 - u. >2014-08-10 14:08:00,800 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 183/331: libXevie.x86_64 0:1.0.3-7.fc20 - u >2014-08-10 14:08:00,809 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 183/331: libXevie.x86_64 0:1.0.3-7.fc20 - u. >2014-08-10 14:08:00,902 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 184/331: libvirt-client.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:08:00,909 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 184/331: libvirt-client.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:08:01,013 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 185/331: libaio.x86_64 0:0.3.109-8.fc20 - u >2014-08-10 14:08:01,025 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 185/331: libaio.x86_64 0:0.3.109-8.fc20 - u. >2014-08-10 14:08:01,147 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 186/331: libwayland-server.x86_64 0:1.2.0-3.fc20 - u >2014-08-10 14:08:01,157 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 186/331: libwayland-server.x86_64 0:1.2.0-3.fc20 - u. >2014-08-10 14:08:01,249 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 187/331: net-snmp-libs.x86_64 1:5.7.2-17.fc20 - u >2014-08-10 14:08:01,257 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 187/331: net-snmp-libs.x86_64 1:5.7.2-17.fc20 - u. >2014-08-10 14:08:01,359 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 188/331: sg3_utils-libs.x86_64 0:1.37-2.fc20 - u >2014-08-10 14:08:01,372 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 188/331: sg3_utils-libs.x86_64 0:1.37-2.fc20 - u. >2014-08-10 14:08:01,485 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 189/331: quota.x86_64 1:4.01-11.fc20 - u >2014-08-10 14:08:01,497 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 189/331: quota.x86_64 1:4.01-11.fc20 - u. >2014-08-10 14:08:01,595 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 190/331: fence-agents-common.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:01,604 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 190/331: fence-agents-common.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:01,722 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 191/331: libvirt-daemon-driver-nodedev.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:08:01,737 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 191/331: libvirt-daemon-driver-nodedev.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:08:01,850 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 192/331: python-pexpect.noarch 0:3.1-1.fc20 - u >2014-08-10 14:08:01,861 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 192/331: python-pexpect.noarch 0:3.1-1.fc20 - u. >2014-08-10 14:08:01,977 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 193/331: numactl.x86_64 0:2.0.9-1.fc20 - u >2014-08-10 14:08:02,002 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 193/331: numactl.x86_64 0:2.0.9-1.fc20 - u. >2014-08-10 14:08:02,111 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 194/331: libXrender.x86_64 0:0.9.8-2.fc20 - u >2014-08-10 14:08:02,118 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 194/331: libXrender.x86_64 0:0.9.8-2.fc20 - u. >2014-08-10 14:08:02,213 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 195/331: SDL.x86_64 0:1.2.15-13.fc20 - u >2014-08-10 14:08:02,225 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 195/331: SDL.x86_64 0:1.2.15-13.fc20 - u. >2014-08-10 14:08:02,340 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 196/331: net-snmp-utils.x86_64 1:5.7.2-17.fc20 - u >2014-08-10 14:08:02,350 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 196/331: net-snmp-utils.x86_64 1:5.7.2-17.fc20 - u. >2014-08-10 14:08:02,449 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 197/331: yum-utils.noarch 0:1.1.31-23.fc20 - u >2014-08-10 14:08:02,457 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 197/331: yum-utils.noarch 0:1.1.31-23.fc20 - u. >2014-08-10 14:08:02,551 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 198/331: libsndfile.x86_64 0:1.0.25-8.fc20 - u >2014-08-10 14:08:02,565 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 198/331: libsndfile.x86_64 0:1.0.25-8.fc20 - u. >2014-08-10 14:08:02,653 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 199/331: syslinux.x86_64 0:4.05-7.fc20 - u >2014-08-10 14:08:02,664 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 199/331: syslinux.x86_64 0:4.05-7.fc20 - u. >2014-08-10 14:08:02,779 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 200/331: perl-Email-Date-Format.noarch 0:1.002-17.fc20 - u >2014-08-10 14:08:02,788 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 200/331: perl-Email-Date-Format.noarch 0:1.002-17.fc20 - u. >2014-08-10 14:08:02,881 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 201/331: perl-Compress-Raw-Bzip2.x86_64 0:2.062-2.fc20 - u >2014-08-10 14:08:02,888 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 201/331: perl-Compress-Raw-Bzip2.x86_64 0:2.062-2.fc20 - u. >2014-08-10 14:08:03,008 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 202/331: yajl.x86_64 0:2.0.4-3.fc20 - u >2014-08-10 14:08:03,020 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 202/331: yajl.x86_64 0:2.0.4-3.fc20 - u. >2014-08-10 14:08:03,142 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 203/331: seabios-bin.noarch 0:1.7.3.1-2.fc20 - u >2014-08-10 14:08:03,153 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 203/331: seabios-bin.noarch 0:1.7.3.1-2.fc20 - u. >2014-08-10 14:08:03,246 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 204/331: libfontenc.x86_64 0:1.1.1-4.fc20 - u >2014-08-10 14:08:03,252 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 204/331: libfontenc.x86_64 0:1.1.1-4.fc20 - u. >2014-08-10 14:08:03,372 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 205/331: cups-libs.x86_64 1:1.7.4-3.fc20 - u >2014-08-10 14:08:03,385 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 205/331: cups-libs.x86_64 1:1.7.4-3.fc20 - u. >2014-08-10 14:08:03,474 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 206/331: glusterfs-libs.x86_64 0:3.5.1-1.fc20 - u >2014-08-10 14:08:03,484 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 206/331: glusterfs-libs.x86_64 0:3.5.1-1.fc20 - u. >2014-08-10 14:08:03,576 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 207/331: perl-Digest-MD5.x86_64 0:2.53-3.fc20 - u >2014-08-10 14:08:03,583 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 207/331: perl-Digest-MD5.x86_64 0:2.53-3.fc20 - u. >2014-08-10 14:08:03,677 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 208/331: libsemanage-python.x86_64 0:2.1.10-14.fc20 - u >2014-08-10 14:08:03,691 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 208/331: libsemanage-python.x86_64 0:2.1.10-14.fc20 - u. >2014-08-10 14:08:03,779 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 209/331: fence-agents-intelmodular.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:03,790 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 209/331: fence-agents-intelmodular.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:03,906 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 210/331: corosync.x86_64 0:2.3.3-2.fc20 - u >2014-08-10 14:08:03,914 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 210/331: corosync.x86_64 0:2.3.3-2.fc20 - u. >2014-08-10 14:08:04,032 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 211/331: radvd.x86_64 0:1.9.9-1.fc20 - u >2014-08-10 14:08:04,039 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 211/331: radvd.x86_64 0:1.9.9-1.fc20 - u. >2014-08-10 14:08:04,134 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 212/331: python-configobj.noarch 0:4.7.2-7.fc20 - u >2014-08-10 14:08:04,146 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 212/331: python-configobj.noarch 0:4.7.2-7.fc20 - u. >2014-08-10 14:08:04,261 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 213/331: brltty.x86_64 0:4.5-9.fc20 - u >2014-08-10 14:08:04,304 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 213/331: brltty.x86_64 0:4.5-9.fc20 - u. >2014-08-10 14:08:04,411 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 214/331: opus.x86_64 0:1.1-1.fc20 - u >2014-08-10 14:08:04,428 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 214/331: opus.x86_64 0:1.1-1.fc20 - u. >2014-08-10 14:08:04,514 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 215/331: emacs-filesystem.noarch 1:24.3-17.fc20 - u >2014-08-10 14:08:04,527 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 215/331: emacs-filesystem.noarch 1:24.3-17.fc20 - u. >2014-08-10 14:08:04,649 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 216/331: libvirt-daemon-config-nwfilter.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:08:04,660 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 216/331: libvirt-daemon-config-nwfilter.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:08:04,776 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 217/331: fence-agents-scsi.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:04,784 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 217/331: fence-agents-scsi.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:04,965 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 218/331: libtirpc.x86_64 0:0.2.4-3.0.fc20 - u >2014-08-10 14:08:04,974 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 218/331: libtirpc.x86_64 0:0.2.4-3.0.fc20 - u. >2014-08-10 14:08:05,067 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 219/331: virt-what.x86_64 0:1.13-2.fc20 - u >2014-08-10 14:08:05,090 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 219/331: virt-what.x86_64 0:1.13-2.fc20 - u. >2014-08-10 14:08:05,202 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 220/331: logrotate.x86_64 0:3.8.7-1.fc20 - u >2014-08-10 14:08:05,214 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 220/331: logrotate.x86_64 0:3.8.7-1.fc20 - u. >2014-08-10 14:08:05,336 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 221/331: libicu.x86_64 0:50.1.2-10.fc20 - u >2014-08-10 14:08:05,380 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 221/331: libicu.x86_64 0:50.1.2-10.fc20 - u. >2014-08-10 14:08:05,439 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 222/331: perl-XML-Parser.x86_64 0:2.41-11.fc20 - u >2014-08-10 14:08:05,454 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 222/331: perl-XML-Parser.x86_64 0:2.41-11.fc20 - u. >2014-08-10 14:08:05,539 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 223/331: fence-agents-docker.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:05,554 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 223/331: fence-agents-docker.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:05,675 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 224/331: perl-Digest.noarch 0:1.17-291.fc20 - u >2014-08-10 14:08:05,686 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 224/331: perl-Digest.noarch 0:1.17-291.fc20 - u. >2014-08-10 14:08:05,794 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 225/331: qemu-kvm-tools.x86_64 2:1.6.2-7.fc20 - u >2014-08-10 14:08:05,802 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 225/331: qemu-kvm-tools.x86_64 2:1.6.2-7.fc20 - u. >2014-08-10 14:08:05,903 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 226/331: alsa-lib.x86_64 0:1.0.27.2-2.fc20 - u >2014-08-10 14:08:05,910 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 226/331: alsa-lib.x86_64 0:1.0.27.2-2.fc20 - u. >2014-08-10 14:08:06,040 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 227/331: numad.x86_64 0:0.5-12.20130814git.fc20 - u >2014-08-10 14:08:06,051 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 227/331: numad.x86_64 0:0.5-12.20130814git.fc20 - u. >2014-08-10 14:08:06,142 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 228/331: perl-Compress-Raw-Zlib.x86_64 0:2.062-2.fc20 - u >2014-08-10 14:08:06,150 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 228/331: perl-Compress-Raw-Zlib.x86_64 0:2.062-2.fc20 - u. >2014-08-10 14:08:06,242 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 229/331: python-backports.x86_64 0:1.0-3.fc20 - u >2014-08-10 14:08:06,249 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 229/331: python-backports.x86_64 0:1.0-3.fc20 - u. >2014-08-10 14:08:06,345 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 230/331: ldns.x86_64 0:1.6.16-6.fc20 - u >2014-08-10 14:08:06,357 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 230/331: ldns.x86_64 0:1.6.16-6.fc20 - u. >2014-08-10 14:08:06,480 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 231/331: zerofree.x86_64 0:1.0.1-13.fc20 - u >2014-08-10 14:08:06,489 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 231/331: zerofree.x86_64 0:1.0.1-13.fc20 - u. >2014-08-10 14:08:06,622 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 232/331: libiscsi.x86_64 0:1.9.0-4.fc20 - u >2014-08-10 14:08:06,638 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 232/331: libiscsi.x86_64 0:1.9.0-4.fc20 - u. >2014-08-10 14:08:06,749 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 233/331: python-inotify.noarch 0:0.9.4-4.fc20 - u >2014-08-10 14:08:06,763 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 233/331: python-inotify.noarch 0:0.9.4-4.fc20 - u. >2014-08-10 14:08:06,855 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 234/331: fence-agents-vmware-soap.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:06,862 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 234/331: fence-agents-vmware-soap.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:06,957 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 235/331: glusterfs-fuse.x86_64 0:3.5.1-1.fc20 - u >2014-08-10 14:08:06,978 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 235/331: glusterfs-fuse.x86_64 0:3.5.1-1.fc20 - u. >2014-08-10 14:08:07,066 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 236/331: perl-HTTP-Cookies.noarch 0:6.01-7.fc20 - u >2014-08-10 14:08:07,077 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 236/331: perl-HTTP-Cookies.noarch 0:6.01-7.fc20 - u. >2014-08-10 14:08:07,168 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 237/331: perl-Net-LibIDN.x86_64 0:0.12-16.fc20 - u >2014-08-10 14:08:07,177 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 237/331: perl-Net-LibIDN.x86_64 0:0.12-16.fc20 - u. >2014-08-10 14:08:07,273 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 238/331: tcp_wrappers.x86_64 0:7.6-76.fc20 - u >2014-08-10 14:08:07,284 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 238/331: tcp_wrappers.x86_64 0:7.6-76.fc20 - u. >2014-08-10 14:08:07,377 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 239/331: libseccomp.x86_64 0:2.1.0-1.fc20 - u >2014-08-10 14:08:07,384 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 239/331: libseccomp.x86_64 0:2.1.0-1.fc20 - u. >2014-08-10 14:08:07,483 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 240/331: libXdamage.x86_64 0:1.1.4-4.fc20 - u >2014-08-10 14:08:07,491 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 240/331: libXdamage.x86_64 0:1.1.4-4.fc20 - u. >2014-08-10 14:08:07,609 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 241/331: icoutils.x86_64 0:0.31.0-3.fc20 - u >2014-08-10 14:08:07,615 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 241/331: icoutils.x86_64 0:0.31.0-3.fc20 - u. >2014-08-10 14:08:07,743 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 242/331: libnl.x86_64 0:1.1.4-3.fc20 - u >2014-08-10 14:08:07,756 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 242/331: libnl.x86_64 0:1.1.4-3.fc20 - u. >2014-08-10 14:08:07,846 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 243/331: mailcap.noarch 0:2.1.42-1.fc20 - u >2014-08-10 14:08:07,864 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 243/331: mailcap.noarch 0:2.1.42-1.fc20 - u. >2014-08-10 14:08:07,984 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 244/331: perl-HTTP-Negotiate.noarch 0:6.01-7.fc20 - u >2014-08-10 14:08:07,996 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 244/331: perl-HTTP-Negotiate.noarch 0:6.01-7.fc20 - u. >2014-08-10 14:08:08,090 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 245/331: gsm.x86_64 0:1.0.13-10.fc20 - u >2014-08-10 14:08:08,096 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 245/331: gsm.x86_64 0:1.0.13-10.fc20 - u. >2014-08-10 14:08:08,192 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 246/331: perl-File-Listing.noarch 0:6.04-7.fc20 - u >2014-08-10 14:08:08,203 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 246/331: perl-File-Listing.noarch 0:6.04-7.fc20 - u. >2014-08-10 14:08:08,317 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 247/331: gtk2.x86_64 0:2.24.22-2.fc20 - u >2014-08-10 14:08:08,327 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 247/331: gtk2.x86_64 0:2.24.22-2.fc20 - u. >2014-08-10 14:08:08,453 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 248/331: poppler-data.noarch 0:0.4.6-4.fc20 - u >2014-08-10 14:08:08,460 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 248/331: poppler-data.noarch 0:0.4.6-4.fc20 - u. >2014-08-10 14:08:08,588 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 249/331: atk.x86_64 0:2.10.0-1.fc20 - u >2014-08-10 14:08:08,601 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 249/331: atk.x86_64 0:2.10.0-1.fc20 - u. >2014-08-10 14:08:08,705 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 250/331: libosinfo.x86_64 0:0.2.9-1.fc20 - u >2014-08-10 14:08:08,717 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 250/331: libosinfo.x86_64 0:0.2.9-1.fc20 - u. >2014-08-10 14:08:09,304 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 251/331: quota-nls.noarch 1:4.01-11.fc20 - u >2014-08-10 14:08:09,313 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 251/331: quota-nls.noarch 1:4.01-11.fc20 - u. >2014-08-10 14:08:09,406 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 252/331: flac-libs.x86_64 0:1.3.0-3.fc20 - u >2014-08-10 14:08:09,420 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 252/331: flac-libs.x86_64 0:1.3.0-3.fc20 - u. >2014-08-10 14:08:09,508 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 253/331: fuse-libs.x86_64 0:2.9.3-2.fc20 - u >2014-08-10 14:08:09,532 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 253/331: fuse-libs.x86_64 0:2.9.3-2.fc20 - u. >2014-08-10 14:08:09,634 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 254/331: numactl-libs.x86_64 0:2.0.9-1.fc20 - u >2014-08-10 14:08:09,652 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 254/331: numactl-libs.x86_64 0:2.0.9-1.fc20 - u. >2014-08-10 14:08:09,736 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 255/331: fence-agents-ilo-mp.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:09,743 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 255/331: fence-agents-ilo-mp.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:09,863 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 256/331: perl-TimeDate.noarch 1:2.30-3.fc20 - u >2014-08-10 14:08:09,884 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 256/331: perl-TimeDate.noarch 1:2.30-3.fc20 - u. >2014-08-10 14:08:09,989 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 257/331: libvorbis.x86_64 1:1.3.4-1.fc20 - u >2014-08-10 14:08:10,000 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 257/331: libvorbis.x86_64 1:1.3.4-1.fc20 - u. >2014-08-10 14:08:10,091 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 258/331: hivex.x86_64 0:1.3.8-3.fc20 - u >2014-08-10 14:08:10,099 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 258/331: hivex.x86_64 0:1.3.8-3.fc20 - u. >2014-08-10 14:08:10,218 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 259/331: fence-agents-bladecenter.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:10,248 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 259/331: fence-agents-bladecenter.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:10,344 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 260/331: hexedit.x86_64 0:1.2.13-4.fc20 - u >2014-08-10 14:08:10,356 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 260/331: hexedit.x86_64 0:1.2.13-4.fc20 - u. >2014-08-10 14:08:10,446 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 261/331: vdsm-jsonrpc.noarch 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:08:10,455 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 261/331: vdsm-jsonrpc.noarch 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:08:10,576 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 262/331: fence-agents-ldom.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:10,588 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 262/331: fence-agents-ldom.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:10,700 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 263/331: fence-agents-cisco-mds.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:10,712 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 263/331: fence-agents-cisco-mds.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:10,828 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 264/331: libvirt-daemon.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:08:10,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 264/331: libvirt-daemon.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:08:10,962 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 265/331: mesa-libEGL.x86_64 0:10.1.5-1.20140607.fc20 - u >2014-08-10 14:08:10,969 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 265/331: mesa-libEGL.x86_64 0:10.1.5-1.20140607.fc20 - u. >2014-08-10 14:08:11,064 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 266/331: xorg-x11-font-utils.x86_64 1:7.5-18.fc20 - u >2014-08-10 14:08:11,077 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 266/331: xorg-x11-font-utils.x86_64 1:7.5-18.fc20 - u. >2014-08-10 14:08:11,169 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 267/331: python-ethtool.x86_64 0:0.10-1.fc20 - u >2014-08-10 14:08:11,184 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 267/331: python-ethtool.x86_64 0:0.10-1.fc20 - u. >2014-08-10 14:08:11,271 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 268/331: fence-agents-drac5.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:11,292 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 268/331: fence-agents-drac5.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:11,385 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 269/331: colord-libs.x86_64 0:1.1.7-1.fc20 - u >2014-08-10 14:08:11,391 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 269/331: colord-libs.x86_64 0:1.1.7-1.fc20 - u. >2014-08-10 14:08:11,512 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 270/331: libusal.x86_64 0:1.1.11-22.fc20 - u >2014-08-10 14:08:11,523 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 270/331: libusal.x86_64 0:1.1.11-22.fc20 - u. >2014-08-10 14:08:11,625 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 271/331: perl-Convert-BinHex.noarch 0:1.122-1.fc20 - u >2014-08-10 14:08:11,639 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 271/331: perl-Convert-BinHex.noarch 0:1.122-1.fc20 - u. >2014-08-10 14:08:11,726 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 272/331: fence-agents-rsb.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:11,755 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 272/331: fence-agents-rsb.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:11,828 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 273/331: seavgabios-bin.noarch 0:1.7.3.1-2.fc20 - u >2014-08-10 14:08:11,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 273/331: seavgabios-bin.noarch 0:1.7.3.1-2.fc20 - u. >2014-08-10 14:08:11,955 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 274/331: libXft.x86_64 0:2.3.1-5.fc20 - u >2014-08-10 14:08:11,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 274/331: libXft.x86_64 0:2.3.1-5.fc20 - u. >2014-08-10 14:08:12,083 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 275/331: cryptopp.x86_64 0:5.6.2-3.fc20 - u >2014-08-10 14:08:12,103 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 275/331: cryptopp.x86_64 0:5.6.2-3.fc20 - u. >2014-08-10 14:08:12,210 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 276/331: celt051.x86_64 0:0.5.1.3-7.fc20 - u >2014-08-10 14:08:12,219 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 276/331: celt051.x86_64 0:0.5.1.3-7.fc20 - u. >2014-08-10 14:08:12,345 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 277/331: mtools.x86_64 0:4.0.18-4.fc20 - u >2014-08-10 14:08:12,368 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 277/331: mtools.x86_64 0:4.0.18-4.fc20 - u. >2014-08-10 14:08:12,446 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 278/331: fence-agents-rsa.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:12,467 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 278/331: fence-agents-rsa.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:12,573 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 279/331: pyparted.x86_64 1:3.9-4.fc20 - u >2014-08-10 14:08:12,592 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 279/331: pyparted.x86_64 1:3.9-4.fc20 - u. >2014-08-10 14:08:12,675 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 280/331: pango.x86_64 0:1.36.1-3.fc20 - u >2014-08-10 14:08:12,691 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 280/331: pango.x86_64 0:1.36.1-3.fc20 - u. >2014-08-10 14:08:12,789 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 281/331: gnutls-utils.x86_64 0:3.1.25-1.fc20 - u >2014-08-10 14:08:12,799 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 281/331: gnutls-utils.x86_64 0:3.1.25-1.fc20 - u. >2014-08-10 14:08:12,889 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 282/331: python-setuptools.noarch 0:1.4.2-1.fc20 - u >2014-08-10 14:08:12,898 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 282/331: python-setuptools.noarch 0:1.4.2-1.fc20 - u. >2014-08-10 14:08:13,041 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 283/331: pm-utils.x86_64 0:1.4.1-26.fc20 - u >2014-08-10 14:08:13,064 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 283/331: pm-utils.x86_64 0:1.4.1-26.fc20 - u. >2014-08-10 14:08:13,144 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 284/331: jasper-libs.x86_64 0:1.900.1-25.fc20 - u >2014-08-10 14:08:13,155 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 284/331: jasper-libs.x86_64 0:1.900.1-25.fc20 - u. >2014-08-10 14:08:13,278 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 285/331: iptables-services.x86_64 0:1.4.19.1-1.fc20 - u >2014-08-10 14:08:13,287 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 285/331: iptables-services.x86_64 0:1.4.19.1-1.fc20 - u. >2014-08-10 14:08:13,388 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 286/331: nmap-ncat.x86_64 2:6.45-1.fc20 - u >2014-08-10 14:08:13,395 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 286/331: nmap-ncat.x86_64 2:6.45-1.fc20 - u. >2014-08-10 14:08:13,492 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 287/331: fence-agents-apc-snmp.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:13,502 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 287/331: fence-agents-apc-snmp.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:13,597 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 288/331: fuse.x86_64 0:2.9.3-2.fc20 - u >2014-08-10 14:08:13,610 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 288/331: fuse.x86_64 0:2.9.3-2.fc20 - u. >2014-08-10 14:08:13,723 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 289/331: perl-SOAP-Transport-TCP.noarch 0:0.715-11.fc20 - u >2014-08-10 14:08:13,734 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 289/331: perl-SOAP-Transport-TCP.noarch 0:0.715-11.fc20 - u. >2014-08-10 14:08:13,825 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 290/331: attr.x86_64 0:2.4.47-3.fc20 - u >2014-08-10 14:08:13,834 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 290/331: attr.x86_64 0:2.4.47-3.fc20 - u. >2014-08-10 14:08:13,930 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 291/331: unbound-libs.x86_64 0:1.4.21-3.fc20 - u >2014-08-10 14:08:13,941 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 291/331: unbound-libs.x86_64 0:1.4.21-3.fc20 - u. >2014-08-10 14:08:14,032 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 292/331: harfbuzz.x86_64 0:0.9.27-1.fc20 - u >2014-08-10 14:08:14,041 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 292/331: harfbuzz.x86_64 0:0.9.27-1.fc20 - u. >2014-08-10 14:08:14,142 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 293/331: at-spi2-core.x86_64 0:2.10.2-1.fc20 - u >2014-08-10 14:08:14,157 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 293/331: at-spi2-core.x86_64 0:2.10.2-1.fc20 - u. >2014-08-10 14:08:14,268 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 294/331: libvirt-daemon-driver-network.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:08:14,281 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 294/331: libvirt-daemon-driver-network.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:08:14,421 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 295/331: perl-URI.noarch 0:1.60-11.fc20 - u >2014-08-10 14:08:14,430 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 295/331: perl-URI.noarch 0:1.60-11.fc20 - u. >2014-08-10 14:08:14,526 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 296/331: cairo-gobject.x86_64 0:1.13.1-0.1.git337ab1f.fc20 - u >2014-08-10 14:08:14,537 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 296/331: cairo-gobject.x86_64 0:1.13.1-0.1.git337ab1f.fc20 - u. >2014-08-10 14:08:14,690 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 297/331: perl-Net-SMTP-SSL.noarch 0:1.01-16.fc20 - u >2014-08-10 14:08:14,703 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 297/331: perl-Net-SMTP-SSL.noarch 0:1.01-16.fc20 - u. >2014-08-10 14:08:14,817 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 298/331: ipmitool.x86_64 0:1.8.13-3.fc20 - u >2014-08-10 14:08:14,827 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 298/331: ipmitool.x86_64 0:1.8.13-3.fc20 - u. >2014-08-10 14:08:14,919 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 299/331: amtterm.x86_64 0:1.3-7.fc20 - u >2014-08-10 14:08:14,977 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 299/331: amtterm.x86_64 0:1.3-7.fc20 - u. >2014-08-10 14:08:15,022 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 300/331: vdsm-cli.noarch 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:08:15,042 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 300/331: vdsm-cli.noarch 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:08:15,135 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 301/331: libXcursor.x86_64 0:1.1.14-2.fc20 - u >2014-08-10 14:08:15,142 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 301/331: libXcursor.x86_64 0:1.1.14-2.fc20 - u. >2014-08-10 14:08:15,237 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 302/331: libnfsidmap.x86_64 0:0.25-8.fc20 - u >2014-08-10 14:08:15,249 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 302/331: libnfsidmap.x86_64 0:0.25-8.fc20 - u. >2014-08-10 14:08:15,363 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 303/331: perl-IO-Socket-IP.noarch 0:0.30-2.fc20 - u >2014-08-10 14:08:15,374 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 303/331: perl-IO-Socket-IP.noarch 0:0.30-2.fc20 - u. >2014-08-10 14:08:15,490 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 304/331: python-dmidecode.x86_64 0:3.10.13-10.fc20 - u >2014-08-10 14:08:15,498 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 304/331: python-dmidecode.x86_64 0:3.10.13-10.fc20 - u. >2014-08-10 14:08:15,595 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 305/331: vdsm-python.x86_64 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:08:15,605 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 305/331: vdsm-python.x86_64 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:08:15,722 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 306/331: hicolor-icon-theme.noarch 0:0.13-1.fc20 - u >2014-08-10 14:08:15,730 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 306/331: hicolor-icon-theme.noarch 0:0.13-1.fc20 - u. >2014-08-10 14:08:15,848 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 307/331: libusb.x86_64 1:0.1.5-3.fc20 - u >2014-08-10 14:08:15,854 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 307/331: libusb.x86_64 1:0.1.5-3.fc20 - u. >2014-08-10 14:08:15,950 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 308/331: libX11.x86_64 0:1.6.1-1.fc20 - u >2014-08-10 14:08:15,962 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 308/331: libX11.x86_64 0:1.6.1-1.fc20 - u. >2014-08-10 14:08:16,069 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 309/331: desktop-file-utils.x86_64 0:0.22-1.fc20 - u >2014-08-10 14:08:16,077 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 309/331: desktop-file-utils.x86_64 0:0.22-1.fc20 - u. >2014-08-10 14:08:16,195 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 310/331: libxml2-python.x86_64 0:2.9.1-2.fc20 - u >2014-08-10 14:08:16,235 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 310/331: libxml2-python.x86_64 0:2.9.1-2.fc20 - u. >2014-08-10 14:08:16,315 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 311/331: libvirt-python.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:08:16,334 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 311/331: libvirt-python.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:08:16,426 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 312/331: genisoimage.x86_64 0:1.1.11-22.fc20 - u >2014-08-10 14:08:16,433 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 312/331: genisoimage.x86_64 0:1.1.11-22.fc20 - u. >2014-08-10 14:08:16,526 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 313/331: perl-HTML-Tagset.noarch 0:3.20-18.fc20 - u >2014-08-10 14:08:16,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 313/331: perl-HTML-Tagset.noarch 0:3.20-18.fc20 - u. >2014-08-10 14:08:16,653 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 314/331: fence-agents-ibmblade.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:16,665 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 314/331: fence-agents-ibmblade.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:16,756 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 315/331: fence-agents-ilo2.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:16,765 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 315/331: fence-agents-ilo2.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:16,881 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 316/331: nfs-utils.x86_64 1:1.3.0-2.1.fc20 - u >2014-08-10 14:08:16,889 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 316/331: nfs-utils.x86_64 1:1.3.0-2.1.fc20 - u. >2014-08-10 14:08:17,008 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 317/331: ghostscript.x86_64 0:9.14-3.fc20 - u >2014-08-10 14:08:17,030 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 317/331: ghostscript.x86_64 0:9.14-3.fc20 - u. >2014-08-10 14:08:17,134 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 318/331: mesa-libGL.x86_64 0:10.1.5-1.20140607.fc20 - u >2014-08-10 14:08:17,146 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 318/331: mesa-libGL.x86_64 0:10.1.5-1.20140607.fc20 - u. >2014-08-10 14:08:17,236 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 319/331: fence-agents-brocade.x86_64 0:4.0.10-1.fc20 - u >2014-08-10 14:08:17,245 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 319/331: fence-agents-brocade.x86_64 0:4.0.10-1.fc20 - u. >2014-08-10 14:08:17,363 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 320/331: libthai.x86_64 0:0.1.20-1.fc20 - u >2014-08-10 14:08:17,369 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 320/331: libthai.x86_64 0:0.1.20-1.fc20 - u. >2014-08-10 14:08:17,497 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 321/331: device-mapper-multipath-libs.x86_64 0:0.4.9-56.fc20 - u >2014-08-10 14:08:17,537 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 321/331: device-mapper-multipath-libs.x86_64 0:0.4.9-56.fc20 - u. >2014-08-10 14:08:17,601 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 322/331: perl-LWP-MediaTypes.noarch 0:6.02-4.fc20 - u >2014-08-10 14:08:17,617 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 322/331: perl-LWP-MediaTypes.noarch 0:6.02-4.fc20 - u. >2014-08-10 14:08:17,703 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 323/331: libvirt-daemon-driver-secret.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:08:17,709 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 323/331: libvirt-daemon-driver-secret.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:08:17,812 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 324/331: xdg-utils.noarch 0:1.1.0-0.24.rc2.fc20 - u >2014-08-10 14:08:17,824 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 324/331: xdg-utils.noarch 0:1.1.0-0.24.rc2.fc20 - u. >2014-08-10 14:08:17,914 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 325/331: perl-IO-HTML.noarch 0:1.001-1.fc20 - u >2014-08-10 14:08:17,924 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 325/331: perl-IO-HTML.noarch 0:1.001-1.fc20 - u. >2014-08-10 14:08:18,016 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 326/331: python-IPy.noarch 0:0.75-6.fc20 - u >2014-08-10 14:08:18,023 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 326/331: python-IPy.noarch 0:0.75-6.fc20 - u. >2014-08-10 14:08:18,142 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 327/331: libvirt-daemon-driver-qemu.x86_64 0:1.1.3.5-2.fc20 - u >2014-08-10 14:08:18,156 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 327/331: libvirt-daemon-driver-qemu.x86_64 0:1.1.3.5-2.fc20 - u. >2014-08-10 14:08:18,278 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 328/331: mesa-libglapi.x86_64 0:10.1.5-1.20140607.fc20 - u >2014-08-10 14:08:18,288 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 328/331: mesa-libglapi.x86_64 0:10.1.5-1.20140607.fc20 - u. >2014-08-10 14:08:18,397 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 329/331: libICE.x86_64 0:1.0.8-6.fc20 - u >2014-08-10 14:08:18,404 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 329/331: libICE.x86_64 0:1.0.8-6.fc20 - u. >2014-08-10 14:08:18,551 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 330/331: python-ioprocess.noarch 0:0.5.0-1.fc20 - u >2014-08-10 14:08:18,561 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 330/331: python-ioprocess.noarch 0:0.5.0-1.fc20 - u. >2014-08-10 14:08:18,677 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Yum Verify: 331/331: perl-MailTools.noarch 0:2.12-4.fc20 - u >2014-08-10 14:08:18,686 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 331/331: perl-MailTools.noarch 0:2.12-4.fc20 - u. >2014-08-10 14:08:19,106 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Misc configuration >2014-08-10 14:08:19,224 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Misc configuration. >2014-08-10 14:08:19,224 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Setting time >2014-08-10 14:08:19,241 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Setting time. >2014-08-10 14:08:20,841 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Enrolling certificate >2014-08-10 14:08:20,897 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Enrolling certificate. >2014-08-10 14:08:23,369 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Transaction commit >2014-08-10 14:08:23,389 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Transaction commit. >2014-08-10 14:08:23,421 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Closing up >2014-08-10 14:08:23,431 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Closing up. >2014-08-10 14:08:24,051 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stopping libvirtd >2014-08-10 14:08:24,101 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stopping libvirtd. >2014-08-10 14:08:24,102 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Starting vdsm >2014-08-10 14:08:24,110 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Starting vdsm. >2014-08-10 14:08:27,759 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Pre-termination >2014-08-10 14:08:27,778 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Pre-termination. >2014-08-10 14:08:27,786 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Retrieving installation logs to: '/home/vvolansk/ovirt-engine/var/log/ovirt-engine/host-deploy/ovirt-20140810140827-galactica.tlv.redhat.com-34ff773f.log' >2014-08-10 14:08:27,795 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Retrieving installation logs to: '/home/vvolansk/ovirt-engine/var/log/ovirt-engine/host-deploy/ovirt-20140810140827-galactica.tlv.redhat.com-34ff773f.log'. >2014-08-10 14:08:28,158 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica.tlv.redhat.com: Stage: Termination >2014-08-10 14:08:28,176 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 34ff773f, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Termination. >2014-08-10 14:08:31,186 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-45) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:08:31,190 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:08:31,191 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:08:34,692 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-5) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:08:34,693 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:08:34,696 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:08:38,196 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-13) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:08:38,197 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:08:38,201 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:08:41,702 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-14) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:08:41,703 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:08:41,706 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:08:45,206 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-24) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:08:45,208 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:08:45,210 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:08:48,711 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-28) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:08:48,712 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:08:48,716 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:08:52,217 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-29) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:08:52,218 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:08:52,221 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:08:55,722 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-26) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:08:55,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:08:55,728 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:08:59,229 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-36) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:08:59,230 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:08:59,233 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:02,734 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-40) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:02,735 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:02,738 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:06,239 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-44) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:06,241 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:06,243 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:09,744 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-1) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:09,745 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:09,749 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:13,249 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-7) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:13,250 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:13,253 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:16,754 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-20) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:16,754 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:16,757 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:20,258 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-10) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:20,259 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:20,262 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:23,763 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-24) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:23,764 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:23,767 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:27,267 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-35) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:27,268 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:27,271 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:30,772 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-32) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:30,773 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:30,776 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:34,277 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-26) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:34,277 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:34,280 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:37,781 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-47) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:37,781 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:37,785 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:41,286 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-37) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:41,286 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:41,289 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:44,790 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-4) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:44,792 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:44,795 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:48,296 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-1) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:48,297 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:48,300 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:51,801 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-11) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:51,801 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:51,804 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:55,305 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-20) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:55,306 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:55,308 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:09:58,809 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-19) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:09:58,810 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:09:58,813 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:02,314 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-23) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:02,314 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:02,317 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:05,818 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-35) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:05,818 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:05,821 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:09,322 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-27) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:09,323 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:09,326 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:12,827 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-41) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:12,828 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:12,831 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:16,332 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-48) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:16,333 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:16,335 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:19,836 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-6) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:19,837 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:19,840 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:23,341 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-2) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:23,341 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:23,344 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:26,845 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-9) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:26,845 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:26,848 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:30,349 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-15) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:30,349 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:30,352 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:33,853 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-20) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:33,853 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:33,857 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:37,358 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-19) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:37,358 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:37,362 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:40,863 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-30) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:40,863 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:40,866 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:44,367 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-33) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:44,368 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:44,371 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:47,872 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-26) Exception during connection: java.lang.InterruptedException > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:996) [rt.jar:1.7.0_25] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:248) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:47,873 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Command PollVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-08-10 14:10:47,876 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-08-10 14:10:48,378 ERROR [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Host installation failed for host 70abf755-e0dc-4d23-9bd7-a40bc21c4314, h2.: org.ovirt.engine.core.bll.VdsCommand$VdsInstallException: Network error during communication with the host > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.configureManagementNetwork(InstallVdsInternalCommand.java:190) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:164) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2239) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:10:48,391 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314, status=NonResponsive, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 521c06f7 >2014-08-10 14:10:48,450 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] FINISH, SetVdsStatusVDSCommand, log id: 521c06f7 >2014-08-10 14:10:48,459 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Correlation ID: 34ff773f, Job ID: fad1c0e2-8080-496c-8358-5105e69ac9a7, Call Stack: null, Custom Event ID: -1, Message: Host h2 installation failed. Network error during communication with the host. >2014-08-10 14:10:48,467 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-12) [34ff773f] Lock freed to object EngineLock [exclusiveLocks= key: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 value: VDS >, sharedLocks= ] >2014-08-10 14:13:38,684 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-8-thread-10) [7c51607] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-10 14:13:38,779 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-8-thread-10) [7c51607] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDS >2014-08-10 14:13:38,781 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-10) [7c51607] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 3ea5180 >2014-08-10 14:13:51,690 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-8-thread-33) [7271accd] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-10 14:13:51,799 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-8-thread-33) [7271accd] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDS >2014-08-10 14:13:51,802 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-33) [7271accd] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 399d1a50 >2014-08-10 14:15:40,838 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-86) Principal admin::internal synchronized >2014-08-10 14:15:40,999 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Attempting to update VMs/Templates Ovf. >2014-08-10 14:17:21,123 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-8-thread-11) [3805220d] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-10 14:17:21,196 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-8-thread-11) [3805220d] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDS >2014-08-10 14:17:21,198 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-11) [3805220d] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: e663b4c >2014-08-10 14:17:37,343 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (http--0.0.0.0-8080-1) [20853978] Lock Acquired to object EngineLock [exclusiveLocks= key: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 value: VDS_FENCE >, sharedLocks= ] >2014-08-10 14:17:37,344 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (org.ovirt.thread.pool-8-thread-43) [20853978] Running command: FenceVdsManualyCommand internal: false. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-10 14:17:37,348 INFO [org.ovirt.engine.core.bll.ClearNonResponsiveVdsVmsCommand] (org.ovirt.thread.pool-8-thread-43) [328fa534] Running command: ClearNonResponsiveVdsVmsCommand internal: true. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDS >2014-08-10 14:17:37,351 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (org.ovirt.thread.pool-8-thread-43) [328fa534] START, UpdateVdsVMsClearedVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314), log id: 2230c1c >2014-08-10 14:17:48,176 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-8-thread-44) [78c94347] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-10 14:17:48,275 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-8-thread-44) [78c94347] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDS >2014-08-10 14:17:48,277 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-44) [78c94347] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 76027b16 >2014-08-10 14:17:55,750 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-8-thread-2) [1207e1a3] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-10 14:17:55,845 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-8-thread-2) [1207e1a3] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDS >2014-08-10 14:17:55,848 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-2) [1207e1a3] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 4dd35acd >2014-08-10 14:18:13,218 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (http--0.0.0.0-8080-1) [400f716c] Failed to Acquire Lock to object EngineLock [exclusiveLocks= key: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 value: VDS_FENCE >, sharedLocks= ] >2014-08-10 14:18:13,219 WARN [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (http--0.0.0.0-8080-1) [400f716c] CanDoAction of action FenceVdsManualy failed. Reasons:VAR__ACTION__MANUAL_FENCE,VAR__TYPE__HOST,POWER_MANAGEMENT_ACTION_ON_ENTITY_ALREADY_IN_PROGRESS >2014-08-10 14:27:37,347 ERROR [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (Transaction Reaper Worker 0) Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand. >2014-08-10 14:34:26,836 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-8-thread-36) [52ab19cc] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-10 14:34:26,951 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-8-thread-36) [52ab19cc] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDS >2014-08-10 14:34:26,954 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-36) [52ab19cc] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 625baa39 >2014-08-10 14:35:19,640 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (http--0.0.0.0-8080-1) [4fcdc6a8] Failed to Acquire Lock to object EngineLock [exclusiveLocks= key: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 value: VDS_FENCE >, sharedLocks= ] >2014-08-10 14:35:19,640 WARN [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (http--0.0.0.0-8080-1) [4fcdc6a8] CanDoAction of action FenceVdsManualy failed. Reasons:VAR__ACTION__MANUAL_FENCE,VAR__TYPE__HOST,POWER_MANAGEMENT_ACTION_ON_ENTITY_ALREADY_IN_PROGRESS >2014-08-10 14:35:36,167 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-8-thread-19) [680aec7d] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-10 14:35:36,266 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-8-thread-19) [680aec7d] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDS >2014-08-10 14:35:36,268 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-19) [680aec7d] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 21152650 >2014-08-10 14:35:50,256 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-8-thread-6) [1c45e48f] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-10 14:35:50,339 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-8-thread-6) [1c45e48f] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDS >2014-08-10 14:35:50,342 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-6) [1c45e48f] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 40e9cbae >2014-08-10 14:35:56,895 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-8-thread-4) [208ccec8] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-10 14:35:56,981 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-8-thread-4) [208ccec8] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 70abf755-e0dc-4d23-9bd7-a40bc21c4314 Type: VDS >2014-08-10 14:35:56,983 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-4) [208ccec8] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 70abf755-e0dc-4d23-9bd7-a40bc21c4314, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 669c5603 >2014-08-10 14:43:47,667 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-10 14:43:47,669 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-10 14:43:47,670 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-10 14:43:47,671 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-10 14:43:47,672 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-10 14:43:47,673 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-10 14:43:47,674 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-10 14:43:47,675 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-10 14:43:47,675 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-10 14:43:47,676 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-10 14:43:47,682 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-10 14:43:47,683 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-10 14:43:47,683 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-10 14:43:47,684 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-10 14:43:47,684 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-10 14:43:47,685 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-10 14:43:47,686 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-10 14:43:47,695 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-10 14:43:47,696 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-10 14:43:47,696 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-10 14:43:47,697 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-10 14:43:47,697 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-10 14:43:47,698 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DB_USER" is "engine". >2014-08-10 14:43:47,698 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-10 14:43:47,699 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-10 14:43:47,699 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-10 14:43:47,700 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-10 14:43:47,701 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-10 14:43:47,701 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-10 14:43:47,702 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-10 14:43:47,702 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-10 14:43:47,703 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-10 14:43:47,703 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-10 14:43:47,704 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-10 14:43:47,704 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-10 14:43:47,706 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-10 14:43:47,707 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-10 14:43:47,707 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-10 14:43:47,710 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-10 14:43:47,711 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-10 14:43:47,715 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-10 14:43:47,716 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-10 14:43:47,716 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-10 14:43:47,717 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-10 14:43:47,717 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-10 14:43:47,718 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-10 14:43:47,719 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-10 14:43:47,725 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-10 14:43:47,725 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-10 14:43:47,726 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-10 14:43:47,726 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-10 14:43:47,727 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-10 14:43:47,727 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-10 14:43:47,728 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-10 14:43:47,728 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-10 14:43:47,729 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-10 14:43:47,729 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-10 14:43:47,730 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-10 14:43:47,730 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-10 14:43:47,737 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-10 14:43:47,737 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-10 14:43:47,738 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-10 14:43:47,738 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-10 14:43:47,739 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_USER" is "ovirt". >2014-08-10 14:43:47,739 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-10 14:43:47,740 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-10 14:43:47,740 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-10 14:43:47,747 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-10 14:43:47,747 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-10 14:43:47,748 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-1) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-10 14:43:48,161 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-6) Start initializing Backend >2014-08-10 14:43:48,295 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-6) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:43:48,399 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-6) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:43:48,445 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-6) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-10 14:43:48,459 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-6) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-10 14:43:48,572 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-6) Running ovirt-engine 3.5.0.0 >2014-08-10 14:43:48,573 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-6) Start initializing dictionaries >2014-08-10 14:43:48,576 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-6) Finished initializing dictionaries >2014-08-10 14:43:48,577 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Start initializing AuditLogCleanupManager >2014-08-10 14:43:48,577 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-10 14:43:48,596 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Finished initializing AuditLogCleanupManager >2014-08-10 14:43:48,597 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Start initializing CommandEntityCleanupManager >2014-08-10 14:43:48,597 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-10 14:43:48,602 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Finished initializing CommandEntityCleanupManager >2014-08-10 14:43:48,606 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-6) Start initializing TagsDirector >2014-08-10 14:43:48,628 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-6) Tag root added to tree >2014-08-10 14:43:48,636 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-6) Finished initializing TagsDirector >2014-08-10 14:43:48,637 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-6) Start initializing IsoDomainListSyncronizer >2014-08-10 14:43:48,648 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-6) Finished initializing IsoDomainListSyncronizer >2014-08-10 14:43:48,750 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-6) Completed initializing handlers >2014-08-10 14:43:48,779 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Start initializing ErrorTranslatorImpl >2014-08-10 14:43:48,796 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-10 14:43:48,797 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Finished initializing ErrorTranslatorImpl >2014-08-10 14:43:48,798 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Start initializing ErrorTranslatorImpl >2014-08-10 14:43:48,799 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Finished initializing ErrorTranslatorImpl >2014-08-10 14:43:48,799 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-6) Mark incomplete jobs as UNKNOWN >2014-08-10 14:43:48,850 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-6) Start initializing JobRepositoryCleanupManager >2014-08-10 14:43:48,851 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-6) Finished initializing JobRepositoryCleanupManager >2014-08-10 14:43:48,852 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-6) Start initializing AutoRecoveryManager >2014-08-10 14:43:48,854 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-6) Finished initializing AutoRecoveryManager >2014-08-10 14:43:48,857 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-6) Start initializing ExecutionMessageDirector >2014-08-10 14:43:48,859 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-6) Finished initializing ExecutionMessageDirector >2014-08-10 14:43:48,911 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-6) Start of enabled extensions list >2014-08-10 14:43:48,911 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-6) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-10 14:43:48,912 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-6) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-10 14:43:48,913 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-6) End of enabled extensions list >2014-08-10 14:43:48,962 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-6) Start initializing DbUserCacheManager >2014-08-10 14:43:48,963 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-6) Finished initializing DbUserCacheManager >2014-08-10 14:43:48,971 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-6) Initialization of AsyncTaskManager completed successfully. >2014-08-10 14:43:48,972 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-6) Start initializing ResourceManager >2014-08-10 14:43:48,997 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-10 14:43:49,057 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-6) Entered VdsManager constructor >2014-08-10 14:43:49,083 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-6) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-10 14:43:49,167 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-6) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-10 14:43:49,183 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-6) Finished initializing ResourceManager >2014-08-10 14:43:49,188 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-6) Initialization of OvfDataUpdater completed successfully. >2014-08-10 14:43:49,189 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Start scheduling to enable vds load balancer >2014-08-10 14:43:49,190 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Finished scheduling to enable vds load balancer >2014-08-10 14:43:49,190 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Start HA Reservation check >2014-08-10 14:43:49,191 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Finished HA Reservation check >2014-08-10 14:43:49,198 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-6) Init VM custom properties utilities >2014-08-10 14:43:49,199 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-6) Init device custom properties utilities >2014-08-10 14:43:49,205 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Initializing Scheduling manager >2014-08-10 14:43:49,229 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-10 14:43:49,230 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-10 14:43:49,231 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-10 14:43:49,240 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) External scheduler disabled, discovery skipped >2014-08-10 14:43:49,241 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Initialized Scheduling manager >2014-08-10 14:43:49,241 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-6) Initializing DWH Heart Beat >2014-08-10 14:43:49,242 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-6) DWH Heart Beat initialized >2014-08-10 14:45:14,837 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-10 14:45:14,838 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-10 14:45:14,839 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-10 14:45:14,840 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-10 14:45:14,842 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-10 14:45:14,843 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-10 14:45:14,844 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-10 14:45:14,845 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-10 14:45:14,846 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-10 14:45:14,846 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-10 14:45:14,847 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-10 14:45:14,848 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-10 14:45:14,848 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-10 14:45:14,849 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-10 14:45:14,849 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-10 14:45:14,850 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-10 14:45:14,851 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-10 14:45:14,851 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-10 14:45:14,852 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-10 14:45:14,852 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-10 14:45:14,853 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-10 14:45:14,853 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-10 14:45:14,854 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_USER" is "engine". >2014-08-10 14:45:14,854 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-10 14:45:14,855 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-10 14:45:14,856 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-10 14:45:14,856 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-10 14:45:14,857 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-10 14:45:14,857 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-10 14:45:14,858 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-10 14:45:14,858 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-10 14:45:14,859 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-10 14:45:14,860 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-10 14:45:14,861 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-10 14:45:14,861 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-10 14:45:14,862 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-10 14:45:14,862 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-10 14:45:14,863 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-10 14:45:14,863 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-10 14:45:14,864 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-10 14:45:14,864 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-10 14:45:14,865 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-10 14:45:14,865 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-10 14:45:14,866 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-10 14:45:14,867 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-10 14:45:14,867 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-10 14:45:14,868 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-10 14:45:14,869 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-10 14:45:14,869 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-10 14:45:14,869 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-10 14:45:14,870 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-10 14:45:14,870 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-10 14:45:14,871 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-10 14:45:14,871 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-10 14:45:14,872 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-10 14:45:14,872 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-10 14:45:14,873 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-10 14:45:14,873 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-10 14:45:14,874 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-10 14:45:14,874 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-10 14:45:14,875 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-10 14:45:14,875 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-10 14:45:14,876 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-10 14:45:14,876 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_USER" is "ovirt". >2014-08-10 14:45:14,877 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-10 14:45:14,877 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-10 14:45:14,878 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-10 14:45:14,878 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-10 14:45:14,879 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-10 14:45:14,879 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-10 14:45:15,202 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-2) Start initializing Backend >2014-08-10 14:45:15,311 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-2) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:45:15,394 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-2) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:45:15,431 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-2) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-10 14:45:15,443 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-2) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-10 14:45:15,569 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-2) Running ovirt-engine 3.5.0.0 >2014-08-10 14:45:15,570 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-2) Start initializing dictionaries >2014-08-10 14:45:15,573 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-2) Finished initializing dictionaries >2014-08-10 14:45:15,573 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Start initializing AuditLogCleanupManager >2014-08-10 14:45:15,574 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-10 14:45:15,586 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Finished initializing AuditLogCleanupManager >2014-08-10 14:45:15,587 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Start initializing CommandEntityCleanupManager >2014-08-10 14:45:15,587 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-10 14:45:15,588 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Finished initializing CommandEntityCleanupManager >2014-08-10 14:45:15,590 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-2) Start initializing TagsDirector >2014-08-10 14:45:15,602 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-2) Tag root added to tree >2014-08-10 14:45:15,613 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-2) Finished initializing TagsDirector >2014-08-10 14:45:15,614 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-2) Start initializing IsoDomainListSyncronizer >2014-08-10 14:45:15,620 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-2) Finished initializing IsoDomainListSyncronizer >2014-08-10 14:45:15,703 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-2) Completed initializing handlers >2014-08-10 14:45:15,731 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Start initializing ErrorTranslatorImpl >2014-08-10 14:45:15,750 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-10 14:45:15,751 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Finished initializing ErrorTranslatorImpl >2014-08-10 14:45:15,751 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Start initializing ErrorTranslatorImpl >2014-08-10 14:45:15,752 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Finished initializing ErrorTranslatorImpl >2014-08-10 14:45:15,753 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-2) Mark incomplete jobs as UNKNOWN >2014-08-10 14:45:15,782 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-2) Start initializing JobRepositoryCleanupManager >2014-08-10 14:45:15,784 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-2) Finished initializing JobRepositoryCleanupManager >2014-08-10 14:45:15,785 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-2) Start initializing AutoRecoveryManager >2014-08-10 14:45:15,789 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-2) Finished initializing AutoRecoveryManager >2014-08-10 14:45:15,792 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-2) Start initializing ExecutionMessageDirector >2014-08-10 14:45:15,794 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-2) Finished initializing ExecutionMessageDirector >2014-08-10 14:45:15,856 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-2) Start of enabled extensions list >2014-08-10 14:45:15,857 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-2) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-10 14:45:15,858 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-2) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-10 14:45:15,859 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-2) End of enabled extensions list >2014-08-10 14:45:15,924 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-2) Start initializing DbUserCacheManager >2014-08-10 14:45:15,926 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-2) Finished initializing DbUserCacheManager >2014-08-10 14:45:15,955 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-2) Initialization of AsyncTaskManager completed successfully. >2014-08-10 14:45:15,958 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-2) Start initializing ResourceManager >2014-08-10 14:45:15,965 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-10 14:45:16,047 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-2) Entered VdsManager constructor >2014-08-10 14:45:16,075 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-2) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-10 14:45:16,174 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-2) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-10 14:45:16,193 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-2) Finished initializing ResourceManager >2014-08-10 14:45:16,199 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-2) Initialization of OvfDataUpdater completed successfully. >2014-08-10 14:45:16,200 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Start scheduling to enable vds load balancer >2014-08-10 14:45:16,203 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Finished scheduling to enable vds load balancer >2014-08-10 14:45:16,207 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Start HA Reservation check >2014-08-10 14:45:16,209 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Finished HA Reservation check >2014-08-10 14:45:16,223 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-2) Init VM custom properties utilities >2014-08-10 14:45:16,224 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-2) Init device custom properties utilities >2014-08-10 14:45:16,229 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Initializing Scheduling manager >2014-08-10 14:45:16,288 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-10 14:45:16,289 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-10 14:45:16,291 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-10 14:45:16,293 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) External scheduler disabled, discovery skipped >2014-08-10 14:45:16,294 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Initialized Scheduling manager >2014-08-10 14:45:16,295 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-2) Initializing DWH Heart Beat >2014-08-10 14:45:16,296 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-2) DWH Heart Beat initialized >2014-08-10 14:45:19,767 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 5a2ca1b9 >2014-08-10 14:45:19,774 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 5a2ca1b9 >2014-08-10 14:45:19,779 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-10 14:45:19,783 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: UNASSIGNED not exist in string table >2014-08-10 14:45:19,784 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-10 14:45:19,784 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-10 14:45:19,785 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-10 14:45:19,785 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-10 14:45:19,786 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-10 14:45:19,786 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-10 14:45:19,787 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-10 14:45:19,788 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-10 14:45:19,789 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-10 14:45:19,789 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-10 14:45:19,790 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STOPPED not exist in string table >2014-08-10 14:45:19,790 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STARTED not exist in string table >2014-08-10 14:45:19,791 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_ERROR not exist in string table >2014-08-10 14:45:19,791 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-10 14:45:19,792 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-10 14:45:19,793 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-10 14:45:19,794 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-10 14:45:19,971 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-10 14:45:19,972 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is running with disabled SELinux. >2014-08-10 14:45:20,228 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h1 >2014-08-10 14:45:20,339 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [3a4c1a90] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-10 14:45:26,221 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-19) [154e41db] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-10 14:45:26,245 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-19) [154e41db] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-10 14:45:26,257 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-19) [154e41db] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-10 14:45:26,323 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) [154e41db] Correlation ID: 154e41db, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-10 14:45:26,422 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-19) [154e41db] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-10 14:45:26,431 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-19) [154e41db] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-10 14:45:26,455 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) [154e41db] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-10 14:45:26,475 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [154e41db] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 4b3a493b >2014-08-10 14:45:26,555 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) [154e41db] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 48 GB of free space >2014-08-10 14:45:26,588 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) [154e41db] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 48 GB of free space >2014-08-10 14:45:26,589 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [154e41db] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-10 14:45:26,595 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [154e41db] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 8bd3209 >2014-08-10 14:45:26,601 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [154e41db] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 8bd3209 >2014-08-10 14:45:26,602 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [154e41db] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 4b3a493b >2014-08-10 14:45:26,602 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [154e41db] Discovered no tasks on Storage Pool Default >2014-08-10 14:45:34,345 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-1) Running command: LoginAdminUserCommand internal: false. >2014-08-10 14:45:34,357 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-10 14:45:38,501 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-1) Running command: LoginUserCommand internal: false. >2014-08-10 14:47:55,875 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8080-1) [20ecb142] Failed to initiate vdsm-id request on host with message Command returned failure code 127 during SSH session 'root@galactica' >2014-08-10 14:47:55,899 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8080-1) [20ecb142] Running command: AddVdsCommand internal: false. Entities affected : ID: 8fc43cd8-abe9-4ee7-834b-6416aed91c63 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-08-10 14:47:55,935 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (http--0.0.0.0-8080-1) [7a1cce42] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 2496ffe0-e756-4ae8-8886-70c640bf0df7 Type: VDS >2014-08-10 14:47:55,949 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (http--0.0.0.0-8080-1) [7a1cce42] Lock freed to object EngineLock [exclusiveLocks= key: 76bcbfe3-3419-4fdb-9b89-d147b7fdef99 value: REGISTER_VDS >, sharedLocks= ] >2014-08-10 14:47:55,951 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (http--0.0.0.0-8080-1) [7a1cce42] START, RemoveVdsVDSCommand(HostName = h2, HostId = 2496ffe0-e756-4ae8-8886-70c640bf0df7), log id: 29ee9806 >2014-08-10 14:47:55,952 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (http--0.0.0.0-8080-1) [7a1cce42] FINISH, RemoveVdsVDSCommand, log id: 29ee9806 >2014-08-10 14:47:55,954 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [7a1cce42] START, AddVdsVDSCommand(HostName = h2, HostId = 2496ffe0-e756-4ae8-8886-70c640bf0df7), log id: 3b0127c3 >2014-08-10 14:47:55,954 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [7a1cce42] AddVds - entered , starting logic to add VDS 2496ffe0-e756-4ae8-8886-70c640bf0df7 >2014-08-10 14:47:55,956 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [7a1cce42] AddVds - VDS 2496ffe0-e756-4ae8-8886-70c640bf0df7 was added, will try to add it to the resource manager >2014-08-10 14:47:55,956 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (http--0.0.0.0-8080-1) [7a1cce42] Entered VdsManager constructor >2014-08-10 14:47:55,963 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (http--0.0.0.0-8080-1) [7a1cce42] Initialize vdsBroker (galactica,54,321) >2014-08-10 14:47:55,964 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (http--0.0.0.0-8080-1) [7a1cce42] VDS 2496ffe0-e756-4ae8-8886-70c640bf0df7 was added to the Resource Manager >2014-08-10 14:47:55,965 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-1) [7a1cce42] FINISH, AddVdsVDSCommand, log id: 3b0127c3 >2014-08-10 14:47:55,968 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [7a1cce42] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h2. >2014-08-10 14:47:56,009 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [7a1cce42] Correlation ID: 20ecb142, Job ID: 9c5183de-dfde-4b99-bbec-10ed91489be8, Call Stack: null, Custom Event ID: -1, Message: Host h2 was added by admin. >2014-08-10 14:47:56,021 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-23) [242fde15] Lock Acquired to object EngineLock [exclusiveLocks= key: 2496ffe0-e756-4ae8-8886-70c640bf0df7 value: VDS >, sharedLocks= ] >2014-08-10 14:47:56,042 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-23) [242fde15] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 2496ffe0-e756-4ae8-8886-70c640bf0df7 Type: VDS >2014-08-10 14:47:56,047 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-23) [242fde15] Before Installation host 2496ffe0-e756-4ae8-8886-70c640bf0df7, h2 >2014-08-10 14:47:56,053 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-23) [242fde15] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 2496ffe0-e756-4ae8-8886-70c640bf0df7, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 33542abc >2014-08-10 14:47:56,067 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-23) [242fde15] FINISH, SetVdsStatusVDSCommand, log id: 33542abc >2014-08-10 14:47:56,136 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-8-thread-23) [242fde15] Installation galactica: Connected to host galactica with SSH key fingerprint: af:6b:19:0c:de:1c:76:eb:a5:97:b3:fa:05:62:84:04 >2014-08-10 14:47:56,149 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-23) [242fde15] Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Connected to host galactica with SSH key fingerprint: af:6b:19:0c:de:1c:76:eb:a5:97:b3:fa:05:62:84:04. >2014-08-10 14:47:56,216 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-8-thread-23) [242fde15] Installation of galactica. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /home/vvolansk/ovirt-engine/var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-08-10 14:47:56,220 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-8-thread-23) SSH execute root@galactica 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-08-10 14:47:56,502 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Initializing >2014-08-10 14:47:56,514 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Initializing. >2014-08-10 14:47:56,523 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Environment setup >2014-08-10 14:47:56,530 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Environment setup. >2014-08-10 14:47:56,531 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Environment packages setup >2014-08-10 14:47:56,538 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Environment packages setup. >2014-08-10 14:47:58,965 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Initializing Host: h2 >2014-08-10 14:48:03,299 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Programs detection >2014-08-10 14:48:03,331 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Programs detection. >2014-08-10 14:48:03,332 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Environment customization >2014-08-10 14:48:03,339 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Environment customization. >2014-08-10 14:48:03,339 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Kdump unsupported >2014-08-10 14:48:03,347 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Kdump unsupported. >2014-08-10 14:48:03,355 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Logs at host located at: '/tmp/ovirt-host-deploy-20140810144756-8gtvuw.log' >2014-08-10 14:48:03,363 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Logs at host located at: '/tmp/ovirt-host-deploy-20140810144756-8gtvuw.log'. >2014-08-10 14:48:03,429 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host galactica reports unique id 4C4C4544-0047-3910-8046-CAC04F46354A >2014-08-10 14:48:03,444 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0047-3910-8046-CAC04F46354A to Host galactica >2014-08-10 14:48:03,502 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Disabling Kdump integration >2014-08-10 14:48:03,513 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Disabling Kdump integration. >2014-08-10 14:48:03,526 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Setup validation >2014-08-10 14:48:03,538 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Setup validation. >2014-08-10 14:48:03,557 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Hardware supports virtualization >2014-08-10 14:48:03,570 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Hardware supports virtualization. >2014-08-10 14:48:04,723 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Transaction setup >2014-08-10 14:48:04,802 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Transaction setup. >2014-08-10 14:48:04,803 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Misc configuration >2014-08-10 14:48:04,821 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Misc configuration. >2014-08-10 14:48:04,822 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Package installation >2014-08-10 14:48:04,830 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Package installation. >2014-08-10 14:48:08,717 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Status: Downloading Packages >2014-08-10 14:48:08,731 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Status: Downloading Packages. >2014-08-10 14:48:11,025 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Download/Verify: vdsm-4.16.1-6.gita4a4614.fc20.x86_64 >2014-08-10 14:48:11,042 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-4.16.1-6.gita4a4614.fc20.x86_64. >2014-08-10 14:48:11,384 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Download/Verify: vdsm-cli-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:48:11,397 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-cli-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:48:11,903 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Download/Verify: vdsm-jsonrpc-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:48:11,936 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-jsonrpc-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:48:12,592 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Download/Verify: vdsm-python-4.16.1-6.gita4a4614.fc20.x86_64 >2014-08-10 14:48:12,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-python-4.16.1-6.gita4a4614.fc20.x86_64. >2014-08-10 14:48:12,764 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Download/Verify: vdsm-python-zombiereaper-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:48:12,789 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-python-zombiereaper-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:48:12,938 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Download/Verify: vdsm-xmlrpc-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:48:12,954 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-xmlrpc-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:48:13,279 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Download/Verify: vdsm-yajsonrpc-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:48:13,294 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Download/Verify: vdsm-yajsonrpc-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:48:13,294 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Status: Check Package Signatures >2014-08-10 14:48:13,302 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Status: Check Package Signatures. >2014-08-10 14:48:13,303 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Status: Running Test Transaction >2014-08-10 14:48:13,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Status: Running Test Transaction. >2014-08-10 14:48:13,340 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Status: Running Transaction >2014-08-10 14:48:13,352 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Status: Running Transaction. >2014-08-10 14:48:13,519 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum install: 1/7: vdsm-python-zombiereaper-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:48:13,525 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 1/7: vdsm-python-zombiereaper-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:48:13,767 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum install: 2/7: vdsm-python-4.16.1-6.gita4a4614.fc20.x86_64 >2014-08-10 14:48:13,774 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 2/7: vdsm-python-4.16.1-6.gita4a4614.fc20.x86_64. >2014-08-10 14:48:14,052 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum install: 3/7: vdsm-xmlrpc-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:48:14,072 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 3/7: vdsm-xmlrpc-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:48:14,302 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum install: 4/7: vdsm-yajsonrpc-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:48:14,312 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 4/7: vdsm-yajsonrpc-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:48:14,551 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum install: 5/7: vdsm-jsonrpc-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:48:14,560 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 5/7: vdsm-jsonrpc-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:48:14,816 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum install: 6/7: vdsm-4.16.1-6.gita4a4614.fc20.x86_64 >2014-08-10 14:48:14,825 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 6/7: vdsm-4.16.1-6.gita4a4614.fc20.x86_64. >2014-08-10 14:48:15,948 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum install: 7/7: vdsm-cli-4.16.1-6.gita4a4614.fc20.noarch >2014-08-10 14:48:15,985 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum install: 7/7: vdsm-cli-4.16.1-6.gita4a4614.fc20.noarch. >2014-08-10 14:48:16,200 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Verify: 1/7: vdsm-yajsonrpc.noarch 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:48:16,233 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 1/7: vdsm-yajsonrpc.noarch 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:48:16,305 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Verify: 2/7: vdsm.x86_64 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:48:16,324 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 2/7: vdsm.x86_64 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:48:16,408 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Verify: 3/7: vdsm-xmlrpc.noarch 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:48:16,415 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 3/7: vdsm-xmlrpc.noarch 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:48:16,508 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Verify: 4/7: vdsm-python-zombiereaper.noarch 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:48:16,515 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 4/7: vdsm-python-zombiereaper.noarch 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:48:16,610 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Verify: 5/7: vdsm-cli.noarch 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:48:16,623 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 5/7: vdsm-cli.noarch 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:48:16,712 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Verify: 6/7: vdsm-python.x86_64 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:48:16,722 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 6/7: vdsm-python.x86_64 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:48:16,813 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Yum Verify: 7/7: vdsm-jsonrpc.noarch 0:4.16.1-6.gita4a4614.fc20 - u >2014-08-10 14:48:16,821 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Yum Verify: 7/7: vdsm-jsonrpc.noarch 0:4.16.1-6.gita4a4614.fc20 - u. >2014-08-10 14:48:17,163 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Misc configuration >2014-08-10 14:48:17,177 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Misc configuration. >2014-08-10 14:48:17,179 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Setting time >2014-08-10 14:48:17,194 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Setting time. >2014-08-10 14:48:19,307 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Enrolling certificate >2014-08-10 14:48:19,339 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Enrolling certificate. >2014-08-10 14:48:20,611 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Transaction commit >2014-08-10 14:48:20,656 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Transaction commit. >2014-08-10 14:48:20,656 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Closing up >2014-08-10 14:48:20,664 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Closing up. >2014-08-10 14:48:21,620 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stopping libvirtd >2014-08-10 14:48:21,755 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stopping libvirtd. >2014-08-10 14:48:21,756 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Starting vdsm >2014-08-10 14:48:21,790 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Starting vdsm. >2014-08-10 14:48:23,045 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Pre-termination >2014-08-10 14:48:23,099 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Pre-termination. >2014-08-10 14:48:23,109 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Retrieving installation logs to: '/home/vvolansk/ovirt-engine/var/log/ovirt-engine/host-deploy/ovirt-20140810144823-galactica-242fde15.log' >2014-08-10 14:48:23,140 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Retrieving installation logs to: '/home/vvolansk/ovirt-engine/var/log/ovirt-engine/host-deploy/ovirt-20140810144823-galactica-242fde15.log'. >2014-08-10 14:48:23,346 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Termination >2014-08-10 14:48:23,373 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242fde15, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Termination. >2014-08-10 14:48:23,373 ERROR [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-8-thread-23) SSH error running command root@galactica:'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True': java.lang.RuntimeException: Unexpected error during execution: /root/.bashrc: line 11: /etc/bash_completion.d/vdsClient: No such file or directory > > at org.ovirt.engine.core.uutils.ssh.SSHDialog.executeCommand(SSHDialog.java:337) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1115) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2239) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:48:23,396 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-8-thread-23) [242fde15] Error during host galactica install: java.lang.RuntimeException: Unexpected error during execution: /root/.bashrc: line 11: /etc/bash_completion.d/vdsClient: No such file or directory > > at org.ovirt.engine.core.uutils.ssh.SSHDialog.executeCommand(SSHDialog.java:337) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1115) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2239) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:48:23,421 ERROR [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-23) [242fde15] Host installation failed for host 2496ffe0-e756-4ae8-8886-70c640bf0df7, h2.: java.lang.RuntimeException: Unexpected error during execution: /root/.bashrc: line 11: /etc/bash_completion.d/vdsClient: No such file or directory > > at org.ovirt.engine.core.uutils.ssh.SSHDialog.executeCommand(SSHDialog.java:337) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1115) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2239) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-10 14:48:23,438 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-23) [242fde15] START, SetVdsStatusVDSCommand(HostName = h2, HostId = 2496ffe0-e756-4ae8-8886-70c640bf0df7, status=InstallFailed, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: ab0bbed >2014-08-10 14:48:23,455 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-23) [242fde15] FINISH, SetVdsStatusVDSCommand, log id: ab0bbed >2014-08-10 14:48:23,488 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-23) [242fde15] Correlation ID: 242fde15, Job ID: 9c5183de-dfde-4b99-bbec-10ed91489be8, Call Stack: null, Custom Event ID: -1, Message: Host h2 installation failed. Unexpected error during execution: /root/.bashrc: line 11: /etc/bash_completion.d/vdsClient: No such file or directory >. >2014-08-10 14:48:23,504 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-23) [242fde15] Lock freed to object EngineLock [exclusiveLocks= key: 2496ffe0-e756-4ae8-8886-70c640bf0df7 value: VDS >, sharedLocks= ] >2014-08-10 14:48:24,697 ERROR [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8080-4) [3684c446] Failed to establish session with host h2: org.apache.sshd.common.RuntimeSshException: Failed to get the session. > at org.apache.sshd.client.future.DefaultConnectFuture.getSession(DefaultConnectFuture.java:43) > at org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:294) [uutils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canConnect(AddVdsCommand.java:443) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand.canDoAction(AddVdsCommand.java:356) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:739) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:341) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:369) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view6.runAction(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runAction(GenericApiGWTServiceImpl.java:134) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] >Caused by: java.net.ConnectException: Connection timed out > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishConnect(UnixAsynchronousSocketChannelImpl.java:252) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:213) [rt.jar:1.7.0_25] > at sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:293) [rt.jar:1.7.0_25] > ... 1 more > >2014-08-10 14:48:24,724 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8080-4) [3684c446] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server galactica.tlv.reshat.com,VDS_CANNOT_CONNECT_TO_SERVER >2014-08-10 14:54:32,916 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (http--0.0.0.0-8080-4) [4279688f] Lock Acquired to object EngineLock [exclusiveLocks= key: 2496ffe0-e756-4ae8-8886-70c640bf0df7 value: VDS >, sharedLocks= ] >2014-08-10 14:54:32,925 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-8-thread-43) [4279688f] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 2496ffe0-e756-4ae8-8886-70c640bf0df7 Type: VDSAction group DELETE_HOST with role type ADMIN >2014-08-10 14:54:32,991 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-8-thread-43) [4279688f] START, RemoveVdsVDSCommand( HostId = 2496ffe0-e756-4ae8-8886-70c640bf0df7), log id: 77a465d2 >2014-08-10 14:54:32,994 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-8-thread-43) [4279688f] vdsManager::disposing >2014-08-10 14:54:32,995 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-8-thread-43) [4279688f] FINISH, RemoveVdsVDSCommand, log id: 77a465d2 >2014-08-10 14:54:33,006 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-43) [4279688f] Correlation ID: 4279688f, Call Stack: null, Custom Event ID: -1, Message: Host h2 was removed by admin. >2014-08-10 14:54:33,007 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-8-thread-43) [4279688f] Lock freed to object EngineLock [exclusiveLocks= key: 2496ffe0-e756-4ae8-8886-70c640bf0df7 value: VDS >, sharedLocks= ] >2014-08-10 14:54:59,113 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (http--0.0.0.0-8080-4) [2dd825c3] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-00000000039f Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-08-10 14:54:59,132 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (http--0.0.0.0-8080-4) [58972fd3] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 Type: VDS >2014-08-10 14:54:59,147 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (http--0.0.0.0-8080-4) [58972fd3] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-00000000037f value: REGISTER_VDS >, sharedLocks= ] >2014-08-10 14:54:59,149 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (http--0.0.0.0-8080-4) [58972fd3] START, RemoveVdsVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6), log id: 599156b1 >2014-08-10 14:54:59,149 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (http--0.0.0.0-8080-4) [58972fd3] FINISH, RemoveVdsVDSCommand, log id: 599156b1 >2014-08-10 14:54:59,151 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-4) [58972fd3] START, AddVdsVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6), log id: 2c5f8cfc >2014-08-10 14:54:59,151 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-4) [58972fd3] AddVds - entered , starting logic to add VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 >2014-08-10 14:54:59,152 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-4) [58972fd3] AddVds - VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added, will try to add it to the resource manager >2014-08-10 14:54:59,153 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (http--0.0.0.0-8080-4) [58972fd3] Entered VdsManager constructor >2014-08-10 14:54:59,159 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (http--0.0.0.0-8080-4) [58972fd3] Initialize vdsBroker (galactica,54,321) >2014-08-10 14:54:59,177 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (http--0.0.0.0-8080-4) [58972fd3] VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-10 14:54:59,178 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (http--0.0.0.0-8080-4) [58972fd3] FINISH, AddVdsVDSCommand, log id: 2c5f8cfc >2014-08-10 14:54:59,180 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [58972fd3] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h2. >2014-08-10 14:54:59,205 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [58972fd3] Correlation ID: 2dd825c3, Job ID: 1f37b759-d2e0-43b8-9270-4f4100cc0cc9, Call Stack: null, Custom Event ID: -1, Message: Host h2 was added by admin. >2014-08-10 14:54:59,213 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-21) [6fed53ec] Lock Acquired to object EngineLock [exclusiveLocks= key: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 value: VDS >, sharedLocks= ] >2014-08-10 14:54:59,229 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-21) [6fed53ec] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 Type: VDS >2014-08-10 14:54:59,230 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-21) [6fed53ec] Before Installation host eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, h2 >2014-08-10 14:54:59,233 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-21) [6fed53ec] START, SetVdsStatusVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 46bfb9b >2014-08-10 14:54:59,246 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-21) [6fed53ec] FINISH, SetVdsStatusVDSCommand, log id: 46bfb9b >2014-08-10 14:54:59,366 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-8-thread-21) [6fed53ec] Installation galactica: Connected to host galactica with SSH key fingerprint: af:6b:19:0c:de:1c:76:eb:a5:97:b3:fa:05:62:84:04 >2014-08-10 14:54:59,378 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-21) [6fed53ec] Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Connected to host galactica with SSH key fingerprint: af:6b:19:0c:de:1c:76:eb:a5:97:b3:fa:05:62:84:04. >2014-08-10 14:54:59,417 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-8-thread-21) [6fed53ec] Installation of galactica. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /home/vvolansk/ovirt-engine/var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-08-10 14:54:59,425 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-8-thread-21) SSH execute root@galactica 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-08-10 14:54:59,685 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Initializing >2014-08-10 14:54:59,693 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Initializing. >2014-08-10 14:54:59,702 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Environment setup >2014-08-10 14:54:59,710 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Environment setup. >2014-08-10 14:54:59,710 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Environment packages setup >2014-08-10 14:54:59,718 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Environment packages setup. >2014-08-10 14:55:02,179 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Initializing Host: h2 >2014-08-10 14:55:04,075 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Programs detection >2014-08-10 14:55:04,108 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Programs detection. >2014-08-10 14:55:04,109 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Environment customization >2014-08-10 14:55:04,141 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Environment customization. >2014-08-10 14:55:04,142 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Kdump unsupported >2014-08-10 14:55:04,157 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Kdump unsupported. >2014-08-10 14:55:04,165 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Logs at host located at: '/tmp/ovirt-host-deploy-20140810145459-a7bkxf.log' >2014-08-10 14:55:04,174 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Logs at host located at: '/tmp/ovirt-host-deploy-20140810145459-a7bkxf.log'. >2014-08-10 14:55:04,237 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host galactica reports unique id 4C4C4544-0047-3910-8046-CAC04F46354A >2014-08-10 14:55:04,247 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0047-3910-8046-CAC04F46354A to Host galactica >2014-08-10 14:55:04,309 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Disabling Kdump integration >2014-08-10 14:55:04,323 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Disabling Kdump integration. >2014-08-10 14:55:04,336 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Setup validation >2014-08-10 14:55:04,348 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Setup validation. >2014-08-10 14:55:04,368 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Hardware supports virtualization >2014-08-10 14:55:04,381 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Hardware supports virtualization. >2014-08-10 14:55:05,590 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Transaction setup >2014-08-10 14:55:05,598 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Transaction setup. >2014-08-10 14:55:05,609 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Misc configuration >2014-08-10 14:55:05,623 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Misc configuration. >2014-08-10 14:55:05,623 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Package installation >2014-08-10 14:55:05,631 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Package installation. >2014-08-10 14:55:15,416 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Misc configuration >2014-08-10 14:55:15,429 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Misc configuration. >2014-08-10 14:55:15,430 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Setting time >2014-08-10 14:55:15,446 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Setting time. >2014-08-10 14:55:18,265 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Enrolling certificate >2014-08-10 14:55:18,304 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Enrolling certificate. >2014-08-10 14:55:19,496 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Transaction commit >2014-08-10 14:55:19,538 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Transaction commit. >2014-08-10 14:55:19,539 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Closing up >2014-08-10 14:55:19,546 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Closing up. >2014-08-10 14:55:20,159 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stopping libvirtd >2014-08-10 14:55:20,192 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stopping libvirtd. >2014-08-10 14:55:20,287 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Starting vdsm >2014-08-10 14:55:20,300 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Starting vdsm. >2014-08-10 14:55:21,525 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Pre-termination >2014-08-10 14:55:21,567 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Pre-termination. >2014-08-10 14:55:21,575 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Retrieving installation logs to: '/home/vvolansk/ovirt-engine/var/log/ovirt-engine/host-deploy/ovirt-20140810145521-galactica-6fed53ec.log' >2014-08-10 14:55:21,584 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Retrieving installation logs to: '/home/vvolansk/ovirt-engine/var/log/ovirt-engine/host-deploy/ovirt-20140810145521-galactica-6fed53ec.log'. >2014-08-10 14:55:21,736 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation galactica: Stage: Termination >2014-08-10 14:55:21,749 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fed53ec, Call Stack: null, Custom Event ID: -1, Message: Installing Host h2. Stage: Termination. >2014-08-10 14:55:21,765 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-10 14:55:23,801 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-10 14:55:23,802 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:cae77ae4-df50-47b2-a1c8-8e487da8529f >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-10 14:55:23,806 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (org.ovirt.thread.pool-8-thread-47) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:94 > >{"jsonrpc":"2.0","method":"Host.ping","params":{},"id":"9a38e25c-843d-471d-866d-52685be9df80"} >2014-08-10 14:55:23,847 INFO [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker) Message received :{"jsonrpc": "2.0", "id": "9a38e25c-843d-471d-866d-52685be9df80", "result": true} >2014-08-10 14:55:23,900 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-8-thread-21) [6fed53ec] Engine managed to communicate with VDSM agent on host h2 >2014-08-10 14:55:23,901 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (org.ovirt.thread.pool-8-thread-21) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"8096375f-6f1c-4084-b9c9-71bc50552617"} >2014-08-10 14:55:24,048 INFO [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker) Message received :{"jsonrpc": "2.0", "id": "8096375f-6f1c-4084-b9c9-71bc50552617", "result": {"HBAInventory": {"iSCSI": [{"InitiatorName": "iqn.1994-05.com.redhat:3a56ae579d41"}], "FC": []}, "packages2": {"kernel": {"release": "200.fc20.x86_64", "buildtime": 1404732281.0, "version": "3.15.4"}, "spice-server": {"release": "2.fc20", "buildtime": 1400513316, "version": "0.12.5"}, "vdsm": {"release": "6.gita4a4614.fc20", "buildtime": 1407229004, "version": "4.16.1"}, "qemu-kvm": {"release": "7.fc20", "buildtime": 1405606060, "version": "1.6.2"}, "libvirt": {"release": "2.fc20", "buildtime": 1400540444, "version": "1.1.3.5"}, "qemu-img": {"release": "7.fc20", "buildtime": 1405606060, "version": "1.6.2"}, "mom": {"release": "1.fc20", "buildtime": 1403118323, "version": "0.4.1"}}, "numaNodeDistance": {"0": [10]}, "cpuModel": "Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz", "hooks": {}, "cpuSockets": "1", "vmTypes": ["kvm"], "selinux": {"mode": "-1"}, "kdumpStatus": 0, "supportedProtocols": ["2.2", "2.3"], "networks": {}, "bridges": {}, "uuid": "4C4C4544-0047-3910-8046-CAC04F46354A", "lastClientIface": "em1", "nics": {"em1": {"addr": "10.35.0.106", "cfg": {"UUID": "6eea9a00-fb47-469d-ab3e-bc426c4966e3", "NETBOOT": "yes", "HWADDR": "84:8f:69:f9:67:60", "BOOTPROTO": "dhcp", "DEVICE": "em1", "TYPE": "Ethernet", "ONBOOT": "yes", "NAME": "em1"}, "ipv6addrs": ["fe80::868f:69ff:fef9:6760/64"], "mtu": "1500", "netmask": "255.255.254.0", "ipv4addrs": ["10.35.0.106/23"], "hwaddr": "84:8f:69:f9:67:60", "speed": 1000}}, "software_revision": "6", "clusterLevels": ["3.0", "3.1", "3.2", "3.3", "3.4", "3.5"], "cpuFlags": "fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,fsgsbase,smep,erms,model_Nehalem,model_Conroe,model_coreduo,model_core2duo,model_Penryn,model_Westmere,model_n270,model_SandyBridge", "ISCSIInitiatorName": "iqn.1994-05.com.redhat:3a56ae579d41", "netConfigDirty": "False", "supportedENGINEs": ["3.0", "3.1", "3.2", "3.3", "3.4", "3.5"], "autoNumaBalancing": 0, "reservedMem": "321", "bondings": {"bond0": {"addr": "", "cfg": {}, "ipv6addrs": [], "mtu": "1500", "netmask": "", "ipv4addrs": [], "hwaddr": "52:99:b3:0f:44:8f", "slaves": [], "opts": {}}}, "software_version": "4.16", "memSize": "15935", "cpuSpeed": "3794.187", "numaNodes": {"0": {"totalMemory": "15935", "cpus": [0, 1, 2, 3, 4, 5, 6, 7]}}, "version_name": "Snow Man", "vlans": {}, "cpuCores": "4", "kvmEnabled": "true", "guestOverhead": "65", "cpuThreads": "8", "emulatedMachines": ["pc", "pc-q35-1.4", "pc-q35-1.5", "q35", "isapc", "pc-0.10", "pc-0.11", "pc-0.12", "pc-0.13", "pc-0.14", "pc-0.15", "pc-1.0", "pc-1.1", "pc-1.2", "pc-1.3", "pc-i440fx-1.4", "pc-i440fx-1.5", "none"], "rngSources": ["random"], "operatingSystem": {"release": "3", "version": "20", "name": "Fedora"}}} >2014-08-10 14:55:24,153 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-8-thread-21) [de04e77] Running command: SetupNetworksCommand internal: true. Entities affected : ID: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-08-10 14:55:24,158 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-8-thread-21) [de04e77] START, SetupNetworksVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[ovirtmgmt {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-00000000037f, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[em1 {id=8853b1cc-0d93-4694-8282-9ddafadc5c83, vdsId=eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, macAddress=84:8f:69:f9:67:60, networkName=ovirtmgmt, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.35.0.106, subnet=255.255.254.0, gateway=null, mtu=1500, bridged=false, speed=1000, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}], > removedNetworks=[], > removedBonds=[]), log id: 7bc8337e >2014-08-10 14:55:24,163 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-8-thread-21) [de04e77] FINISH, SetupNetworksVDSCommand, log id: 7bc8337e >2014-08-10 14:55:24,164 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (org.ovirt.thread.pool-8-thread-48) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:94 > >{"jsonrpc":"2.0","method":"Host.ping","params":{},"id":"8155cb73-f686-4e6a-8b95-0ffaa1e53e9c"} >2014-08-10 14:55:24,168 INFO [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker) Message received :{"jsonrpc": "2.0", "id": "8155cb73-f686-4e6a-8b95-0ffaa1e53e9c", "result": true} >2014-08-10 14:55:24,184 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (org.ovirt.thread.pool-8-thread-1) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:300 > >{"jsonrpc":"2.0","method":"Host.setupNetworks","params":{"networks":{"ovirtmgmt":{"mtu":"1500","bootproto":"dhcp","bridged":"true","nic":"em1","defaultRoute":true,"STP":"no"}},"bondings":{},"options":{"connectivityTimeout":120,"connectivityCheck":"true"}},"id":"ad16176c-77f0-4088-9fc5-89e41fa7df16"} >2014-08-10 14:58:24,795 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:cae77ae4-df50-47b2-a1c8-8e487da8529f > > >2014-08-10 14:58:24,795 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:e5ddf786-7e6b-49c9-90f3-972de4fc1400 > > >2014-08-10 14:58:24,796 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-8-thread-21) [de04e77] org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-10 14:58:24,800 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-8-thread-21) [de04e77] Command SetupNetworksVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[ovirtmgmt {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-00000000037f, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[em1 {id=8853b1cc-0d93-4694-8282-9ddafadc5c83, vdsId=eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, macAddress=84:8f:69:f9:67:60, networkName=ovirtmgmt, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.35.0.106, subnet=255.255.254.0, gateway=null, mtu=1500, bridged=false, speed=1000, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}], > removedNetworks=[], > removedBonds=[]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-10 14:58:24,802 ERROR [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-8-thread-21) [de04e77] Command org.ovirt.engine.core.bll.network.host.SetupNetworksCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' (Failed with error VDS_NETWORK_ERROR and code 5022) >2014-08-10 14:58:24,853 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-21) [de04e77] Failed to configure management network: Failed to configure management network on host h2 due to setup networks failure. >2014-08-10 14:58:24,855 ERROR [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-21) [de04e77] Host installation failed for host eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, h2.: org.ovirt.engine.core.bll.VdsCommand$VdsInstallException: Failed to configure management network on the host > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.configureManagementNetwork(InstallVdsInternalCommand.java:206) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:164) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2239) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] >Caused by: org.ovirt.engine.core.bll.network.NetworkConfigurator$NetworkConfiguratorException: Failed to configure management network > at org.ovirt.engine.core.bll.network.NetworkConfigurator.configureManagementNetwork(NetworkConfigurator.java:255) [bll.jar:] > at org.ovirt.engine.core.bll.network.NetworkConfigurator.createManagementNetworkIfRequired(NetworkConfigurator.java:82) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.configureManagementNetwork(InstallVdsInternalCommand.java:198) [bll.jar:] > ... 64 more > >2014-08-10 14:58:24,869 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-21) [de04e77] START, SetVdsStatusVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, status=NonOperational, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: ec83aa1 >2014-08-10 14:58:24,878 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-21) [de04e77] FINISH, SetVdsStatusVDSCommand, log id: ec83aa1 >2014-08-10 14:58:24,886 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-21) [de04e77] Correlation ID: 6fed53ec, Job ID: 1f37b759-d2e0-43b8-9270-4f4100cc0cc9, Call Stack: null, Custom Event ID: -1, Message: Host h2 installation failed. Failed to configure management network on the host. >2014-08-10 14:58:24,894 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-8-thread-21) [de04e77] Lock freed to object EngineLock [exclusiveLocks= key: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 value: VDS >, sharedLocks= ] >2014-08-10 14:59:44,086 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-8-thread-29) [486aace6] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-10 14:59:44,088 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-29) [486aace6] START, SetVdsStatusVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 7ba67ae8 >2014-08-10 14:59:44,258 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-29) [486aace6] FINISH, SetVdsStatusVDSCommand, log id: 7ba67ae8 >2014-08-10 14:59:44,357 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-8-thread-29) [486aace6] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 Type: VDS >2014-08-10 14:59:44,366 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-29) [486aace6] Correlation ID: 486aace6, Job ID: 67861de7-5728-4392-848a-163bfeb7423b, Call Stack: null, Custom Event ID: -1, Message: Host h2 was switched to Maintenance mode by admin. >2014-08-10 14:59:44,452 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-10 14:59:45,471 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-10 14:59:45,471 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:63d967cc-108d-454b-b205-f8d9159550d6 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-10 14:59:46,455 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-47) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:98 > >{"jsonrpc":"2.0","method":"Host.getStats","params":{},"id":"b1b06d5b-ccec-4e06-829f-82634d2aff91"} >2014-08-10 14:59:52,595 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-10 15:02:46,470 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:63d967cc-108d-454b-b205-f8d9159550d6 > > >2014-08-10 15:02:46,470 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetStatsVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-10 15:02:46,471 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:6936dd54-ae1d-4dde-a332-d17fe26534f7 > > >2014-08-10 15:02:46,472 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h2 is not responding. It will stay in Connecting state for a grace period of $60 seconds and after that an attempt to fence the host will be issued. >2014-08-10 15:02:46,473 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) vds::refreshVdsStats Failed getVdsStats, vds = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 : h2, error = VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-10 15:02:46,473 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand.executeVdsBrokerCommand(GetStatsVDSCommand.java:24) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsStats(VdsUpdateRunTimeInfo.java:669) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:623) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-10 15:02:46,479 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Failed to refresh VDS , vds = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 : h2, VDS Network Error, continuing. >VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-10 15:02:49,482 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-10 15:02:50,499 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-10 15:02:50,500 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:05e66f33-eb9a-40a0-9c1f-9c436eb3ed97 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-10 15:02:51,485 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-21) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:98 > >{"jsonrpc":"2.0","method":"Host.getStats","params":{},"id":"d628b7c8-f227-4235-889f-3279e52e5735"} >2014-08-10 15:02:53,049 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-10 15:04:53,783 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-18) Command SpmStatusVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-10 15:04:53,785 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-18) [274ac916] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-10 15:04:53,787 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-18) [274ac916] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-10 15:04:53,788 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-18) [274ac916] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-10 15:04:53,979 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-18) [274ac916] Correlation ID: 274ac916, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host h1, Error: Network error during communication with the Host.). >2014-08-11 08:03:50,595 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:05e66f33-eb9a-40a0-9c1f-9c436eb3ed97 > > >2014-08-11 08:03:50,597 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetStatsVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:03:50,608 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:1ca47ca5-7f66-4294-9da7-9989f4d7bcec > > >2014-08-11 08:03:50,617 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Host h2 is not responding. It will stay in Connecting state for a grace period of $60 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:03:50,622 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) vds::refreshVdsStats Failed getVdsStats, vds = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 : h2, error = VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:03:50,629 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand.executeVdsBrokerCommand(GetStatsVDSCommand.java:24) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsStats(VdsUpdateRunTimeInfo.java:669) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:623) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:03:50,641 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-89) Attempting to update VMs/Templates Ovf. >2014-08-11 08:03:50,632 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-88) Start deleteAgedOutCommandEntities >2014-08-11 08:03:50,631 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-85) Start deleteAgedOutAuditLogs >2014-08-11 08:03:50,676 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-90) Principal admin::internal synchronized >2014-08-11 08:03:50,749 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Failed to refresh VDS , vds = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 : h2, VDS Network Error, continuing. >VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:03:50,824 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-88) Finished deleteAgedOutCommandEntities >2014-08-11 08:03:51,053 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-85) Finished deleteAgedOutAuditLogs >2014-08-11 08:03:51,076 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-4) [7cd40bb9] Running command: LogoutUserCommand internal: false. >2014-08-11 08:03:51,076 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-3) [13b5fff9] Running command: LogoutUserCommand internal: false. >2014-08-11 08:03:51,076 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-1) [2105d4ff] Running command: LogoutUserCommand internal: false. >2014-08-11 08:03:51,076 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-2) [454ddeb5] Running command: LogoutUserCommand internal: false. >2014-08-11 08:03:51,079 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-5) [66e54f74] Running command: LogoutUserCommand internal: false. >2014-08-11 08:03:51,178 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [2105d4ff] Correlation ID: 2105d4ff, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-11 08:03:51,228 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [454ddeb5] Correlation ID: 454ddeb5, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-11 08:03:51,228 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [7cd40bb9] Correlation ID: 7cd40bb9, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-11 08:03:51,231 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-3) [13b5fff9] Correlation ID: 13b5fff9, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-11 08:03:51,232 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-5) [66e54f74] Correlation ID: 66e54f74, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-11 08:03:53,787 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.UnknownHostException: galactica > at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) [rt.jar:1.7.0_25] > at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:894) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1286) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName0(InetAddress.java:1239) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:03:53,789 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.UnknownHostException: galactica > at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) [rt.jar:1.7.0_25] > at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:894) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1286) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName0(InetAddress.java:1239) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:03:53,792 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-12) Exception during connection: java.util.concurrent.ExecutionException: java.net.UnknownHostException: galactica > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getVdsStats(JsonRpcVdsServer.java:191) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand.executeVdsBrokerCommand(GetStatsVDSCommand.java:23) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsStats(VdsUpdateRunTimeInfo.java:669) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:623) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: galactica > at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) [rt.jar:1.7.0_25] > at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:894) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1286) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName0(InetAddress.java:1239) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:03:53,799 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetStatsVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: galactica >2014-08-11 08:03:53,801 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Host h2 is not responding. It will stay in Connecting state for a grace period of $60 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:03:53,836 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Server failed to respond, vds_id = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds_name = h2, vm_count = 0, spm_status = None, non-responsive_timeout (seconds) = 60, error = java.net.UnknownHostException: galactica >2014-08-11 08:03:53,894 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: galactica > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsStats(VdsUpdateRunTimeInfo.java:669) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:623) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.UnknownHostException: galactica > at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) > at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:894) > at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1286) > at java.net.InetAddress.getAllByName0(InetAddress.java:1239) > at java.net.InetAddress.getAllByName(InetAddress.java:1155) > at java.net.InetAddress.getAllByName(InetAddress.java:1091) > at java.net.InetAddress.getByName(InetAddress.java:1041) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) >, Custom Event ID: -1, Message: Host h2 is non responsive. >2014-08-11 08:03:53,900 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) vds::refreshVdsStats Failed getVdsStats, vds = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 : h2, error = VDSNetworkException: java.net.UnknownHostException: galactica >2014-08-11 08:03:53,900 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-8-thread-50) ResourceManager::vdsNotResponding entered for Host eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, galactica >2014-08-11 08:03:53,900 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: galactica > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsStats(VdsUpdateRunTimeInfo.java:669) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:623) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: galactica > at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) [rt.jar:1.7.0_25] > at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:894) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1286) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName0(InetAddress.java:1239) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:03:53,968 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-50) [6dabcddf] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 Type: VDS >2014-08-11 08:03:53,978 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-50) [6dabcddf] Opening SSH Soft Fencing session on host galactica >2014-08-11 08:03:53,985 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-50) [6dabcddf] SSH Soft Fencing command failed on host galactica: null >Stdout: >Stderr: >Stacktrace: java.nio.channels.UnresolvedAddressException: java.nio.channels.UnresolvedAddressException > at sun.nio.ch.Net.checkAddress(Net.java:127) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.implConnect(UnixAsynchronousSocketChannelImpl.java:301) [rt.jar:1.7.0_25] > at sun.nio.ch.AsynchronousSocketChannelImpl.connect(AsynchronousSocketChannelImpl.java:208) [rt.jar:1.7.0_25] > at org.apache.sshd.common.io.nio2.Nio2Connector.connect(Nio2Connector.java:47) > at org.apache.sshd.SshClient.connect(SshClient.java:334) > at org.apache.sshd.SshClient.connect(SshClient.java:320) > at org.apache.sshd.SshClient.connect(SshClient.java:310) > at org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:284) [uutils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:87) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:53) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$4.run(VdsEventListener.java:240) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 08:03:54,052 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-50) [196051eb] Lock Acquired to object EngineLock [exclusiveLocks= key: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 value: VDS_FENCE >, sharedLocks= ] >2014-08-11 08:03:54,053 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-50) [196051eb] Failed to run Fence script on vds:h2. >2014-08-11 08:03:54,076 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-50) [196051eb] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h2 became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-08-11 08:03:54,077 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-50) [196051eb] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-08-11 08:03:54,077 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-50) [196051eb] Lock freed to object EngineLock [exclusiveLocks= key: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 value: VDS_FENCE >, sharedLocks= ] >2014-08-11 08:03:56,964 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.UnknownHostException: galactica > at java.net.InetAddress.getAllByName0(InetAddress.java:1243) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:03:56,967 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.UnknownHostException: galactica > at java.net.InetAddress.getAllByName0(InetAddress.java:1243) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:03:56,969 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-22) Exception during connection: java.util.concurrent.ExecutionException: java.net.UnknownHostException: galactica > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: galactica > at java.net.InetAddress.getAllByName0(InetAddress.java:1243) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:03:56,976 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: galactica >2014-08-11 08:03:56,977 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:03:56,991 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: galactica > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: galactica > at java.net.InetAddress.getAllByName0(InetAddress.java:1243) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:04:00,020 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.UnknownHostException: galactica > at java.net.InetAddress.getAllByName0(InetAddress.java:1243) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:04:00,022 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.UnknownHostException: galactica > at java.net.InetAddress.getAllByName0(InetAddress.java:1243) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:04:00,024 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-25) Exception during connection: java.util.concurrent.ExecutionException: java.net.UnknownHostException: galactica > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: galactica > at java.net.InetAddress.getAllByName0(InetAddress.java:1243) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:04:00,030 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: galactica >2014-08-11 08:04:00,031 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:04:00,088 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: galactica > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: galactica > at java.net.InetAddress.getAllByName0(InetAddress.java:1243) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:04:03,114 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.UnknownHostException: galactica > at java.net.InetAddress.getAllByName0(InetAddress.java:1243) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:04:03,116 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.UnknownHostException: galactica > at java.net.InetAddress.getAllByName0(InetAddress.java:1243) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:04:03,118 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-34) Exception during connection: java.util.concurrent.ExecutionException: java.net.UnknownHostException: galactica > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: galactica > at java.net.InetAddress.getAllByName0(InetAddress.java:1243) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:04:03,126 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: galactica >2014-08-11 08:04:03,127 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:04:03,144 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: galactica > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: galactica > at java.net.InetAddress.getAllByName0(InetAddress.java:1243) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1155) [rt.jar:1.7.0_25] > at java.net.InetAddress.getAllByName(InetAddress.java:1091) [rt.jar:1.7.0_25] > at java.net.InetAddress.getByName(InetAddress.java:1041) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:04:04,066 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-6) [3a4c1a90] Command ListVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketException: Connection reset >2014-08-11 08:04:04,066 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) [3a4c1a90] Host h1 is not responding. It will stay in Connecting state for a grace period of $81 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:04:04,067 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) [3a4c1a90] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketException: Connection reset > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:407) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.fetchRunningVms(VdsUpdateRunTimeInfo.java:971) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:920) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:638) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketException: Connection reset > at java.net.SocketInputStream.read(SocketInputStream.java:189) [rt.jar:1.7.0_25] > at java.net.SocketInputStream.read(SocketInputStream.java:121) [rt.jar:1.7.0_25] > at sun.security.ssl.InputRecord.readFully(InputRecord.java:442) [jsse.jar:1.7.0_25] > at sun.security.ssl.InputRecord.read(InputRecord.java:480) [jsse.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927) [jsse.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884) [jsse.jar:1.7.0_25] > at sun.security.ssl.AppInputStream.read(AppInputStream.java:102) [jsse.jar:1.7.0_25] > at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) [rt.jar:1.7.0_25] > at java.io.BufferedInputStream.read(BufferedInputStream.java:254) [rt.jar:1.7.0_25] > at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1413) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.list(Unknown Source) > at sun.reflect.GeneratedMethodAccessor97.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 08:04:04,097 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) [3a4c1a90] Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.net.SocketException: Connection reset >2014-08-11 08:04:06,157 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:04:07,205 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-42) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 339a7447 >2014-08-11 08:04:07,211 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-42) FINISH, GetHardwareInfoVDSCommand, log id: 339a7447 >2014-08-11 08:04:07,259 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-42) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-11 08:04:07,260 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) Host h1 is running with disabled SELinux. >2014-08-11 08:04:07,295 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-42) [21aa9014] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-11 08:04:07,306 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-42) [f022cb8] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-11 08:04:07,346 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-42) [308377eb] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-11 08:04:07,393 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-42) [308377eb] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = NFS, connectionList = [{ id: 1670ace3-7db6-4c73-9b3c-614dcc85bd46, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc5, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc4, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 70964541 >2014-08-11 08:04:07,417 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-42) [308377eb] FINISH, ConnectStorageServerVDSCommand, return: {1670ace3-7db6-4c73-9b3c-614dcc85bd46=0, 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a=0}, log id: 70964541 >2014-08-11 08:04:07,418 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-42) [308377eb] Host h1 storage connection was succeeded >2014-08-11 08:04:07,426 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-16) START, ConnectStoragePoolVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vdsId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, masterVersion = 1), log id: 729430a3 >2014-08-11 08:04:07,872 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-16) FINISH, ConnectStoragePoolVDSCommand, log id: 729430a3 >2014-08-11 08:04:08,091 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-42) [308377eb] START, SetMOMPolicyParametersVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 434e0d2e >2014-08-11 08:04:08,141 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-42) [308377eb] FINISH, SetMOMPolicyParametersVDSCommand, log id: 434e0d2e >2014-08-11 08:04:08,177 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:04:08,177 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:f85e618f-5d3a-4e8c-aeb6-8145025c0746 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:04:08,195 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-42) [308377eb] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host h1 was set to Up. >2014-08-11 08:04:08,199 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-42) [308377eb] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h1. >2014-08-11 08:04:08,229 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-42) [560649b3] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-11 08:04:08,692 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.write0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:94) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.write(IOUtil.java:65) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:466) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:80) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:71) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:95) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:04:09,162 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-41) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"7db4e744-77a9-40e1-a847-98fbcf68407b"} >2014-08-11 08:04:10,337 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) [274ac916] IrsBroker::Failed::GetStoragePoolInfoVDS due to: SocketException: Connection reset >2014-08-11 08:04:20,571 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-73) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 52 GB of free space >2014-08-11 08:04:20,596 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-73) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 52 GB of free space >2014-08-11 08:07:09,178 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:f85e618f-5d3a-4e8c-aeb6-8145025c0746 > > >2014-08-11 08:07:09,178 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@5eaacf9e >2014-08-11 08:07:09,179 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:99b62cb4-f749-4dd1-83ea-e91a7e9c1f33 > > >2014-08-11 08:07:09,179 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) HostName = h2 >2014-08-11 08:07:09,180 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:07:09,180 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:07:09,196 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:07:12,208 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:07:13,228 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:07:13,228 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:d7485c05-0fc0-4699-b17c-9d66f6afe910 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:07:13,244 INFO [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker) Message received :{"jsonrpc": "2.0", "id": "7db4e744-77a9-40e1-a847-98fbcf68407b", "result": {"HBAInventory": {"iSCSI": [{"InitiatorName": "iqn.1994-05.com.redhat:3a56ae579d41"}], "FC": []}, "packages2": {"kernel": {"release": "200.fc20.x86_64", "buildtime": 1404732281.0, "version": "3.15.4"}, "spice-server": {"release": "2.fc20", "buildtime": 1400513316, "version": "0.12.5"}, "vdsm": {"release": "6.gita4a4614.fc20", "buildtime": 1407229004, "version": "4.16.1"}, "qemu-kvm": {"release": "7.fc20", "buildtime": 1405606060, "version": "1.6.2"}, "libvirt": {"release": "2.fc20", "buildtime": 1400540444, "version": "1.1.3.5"}, "qemu-img": {"release": "7.fc20", "buildtime": 1405606060, "version": "1.6.2"}, "mom": {"release": "1.fc20", "buildtime": 1403118323, "version": "0.4.1"}}, "numaNodeDistance": {"0": [10]}, "cpuModel": "Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz", "hooks": {}, "cpuSockets": "1", "vmTypes": ["kvm"], "selinux": {"mode": "-1"}, "kdumpStatus": 0, "supportedProtocols": ["2.2", "2.3"], "networks": {}, "bridges": {}, "uuid": "4C4C4544-0047-3910-8046-CAC04F46354A", "lastClientIface": "em1", "nics": {"em1": {"addr": "10.35.0.106", "cfg": {"UUID": "6eea9a00-fb47-469d-ab3e-bc426c4966e3", "NETBOOT": "yes", "HWADDR": "84:8f:69:f9:67:60", "BOOTPROTO": "dhcp", "DEVICE": "em1", "TYPE": "Ethernet", "ONBOOT": "yes", "NAME": "em1"}, "ipv6addrs": ["fe80::868f:69ff:fef9:6760/64"], "mtu": "1500", "netmask": "255.255.254.0", "ipv4addrs": ["10.35.0.106/23"], "hwaddr": "84:8f:69:f9:67:60", "speed": 1000}}, "software_revision": "6", "clusterLevels": ["3.0", "3.1", "3.2", "3.3", "3.4", "3.5"], "cpuFlags": "fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,fsgsbase,smep,erms,model_Nehalem,model_Conroe,model_coreduo,model_core2duo,model_Penryn,model_Westmere,model_n270,model_SandyBridge", "ISCSIInitiatorName": "iqn.1994-05.com.redhat:3a56ae579d41", "netConfigDirty": "True", "supportedENGINEs": ["3.0", "3.1", "3.2", "3.3", "3.4", "3.5"], "autoNumaBalancing": 0, "reservedMem": "321", "bondings": {"bond0": {"addr": "", "cfg": {}, "ipv6addrs": [], "mtu": "1500", "netmask": "", "ipv4addrs": [], "hwaddr": "52:99:b3:0f:44:8f", "slaves": [], "opts": {}}}, "software_version": "4.16", "memSize": "15935", "cpuSpeed": "1599.726", "numaNodes": {"0": {"totalMemory": "15935", "cpus": [0, 1, 2, 3, 4, 5, 6, 7]}}, "version_name": "Snow Man", "vlans": {}, "cpuCores": "4", "kvmEnabled": "true", "guestOverhead": "65", "cpuThreads": "8", "emulatedMachines": ["pc", "pc-q35-1.4", "pc-q35-1.5", "q35", "isapc", "pc-0.10", "pc-0.11", "pc-0.12", "pc-0.13", "pc-0.14", "pc-0.15", "pc-1.0", "pc-1.1", "pc-1.2", "pc-1.3", "pc-i440fx-1.4", "pc-i440fx-1.5", "none"], "rngSources": ["random"], "operatingSystem": {"release": "3", "version": "20", "name": "Fedora"}}} >2014-08-11 08:07:14,211 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-99) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"d97acf0f-66b5-47eb-8ec6-b272c99d6a54"} >2014-08-11 08:10:14,262 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:d7485c05-0fc0-4699-b17c-9d66f6afe910 > > >2014-08-11 08:10:14,262 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@73b9e6a1 >2014-08-11 08:10:14,263 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:c43d95e1-d522-4b48-ad2f-a0b6dc465da7 > > >2014-08-11 08:10:14,263 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) HostName = h2 >2014-08-11 08:10:14,264 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:10:14,265 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:10:14,281 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:10:17,289 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:10:18,306 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:10:18,307 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:86107a5d-87a8-4cb5-a18c-53d155f01428 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:10:19,291 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-62) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"49a4249f-fa66-4ead-9666-f112e5c0860b"} >2014-08-11 08:10:19,593 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:13:19,304 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:86107a5d-87a8-4cb5-a18c-53d155f01428 > > >2014-08-11 08:13:19,305 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@231b73db >2014-08-11 08:13:19,305 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:0f4b1393-9b76-498d-83e8-1cebfe884fff > > >2014-08-11 08:13:19,305 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) HostName = h2 >2014-08-11 08:13:19,306 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:13:19,307 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:13:19,361 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:13:22,370 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:13:23,390 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:13:23,390 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:22e6df8e-eb1d-4980-aca9-d6eb30974e9f >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:13:24,372 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-20) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"25dfee49-d5ab-45f5-a9e3-7ba3981bacb4"} >2014-08-11 08:13:30,080 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:16:24,388 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:22e6df8e-eb1d-4980-aca9-d6eb30974e9f > > >2014-08-11 08:16:24,388 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:ef832436-1ad7-4a03-9e89-b36f89b616c9 > > >2014-08-11 08:16:24,389 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@7caa6fc9 >2014-08-11 08:16:24,390 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) HostName = h2 >2014-08-11 08:16:24,390 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:16:24,391 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:16:24,434 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:16:27,444 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:16:28,461 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:16:28,461 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:985bdaa8-397b-4d87-b77e-66dca4df6b53 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:16:29,446 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-79) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"7a770526-4a5a-4b27-b9d5-049d49694729"} >2014-08-11 08:16:30,517 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:19:29,461 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:985bdaa8-397b-4d87-b77e-66dca4df6b53 > > >2014-08-11 08:19:29,461 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@607db0cd >2014-08-11 08:19:29,462 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:8044b8e8-a3de-443d-8f94-2d89ff91a3c0 > > >2014-08-11 08:19:29,463 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) HostName = h2 >2014-08-11 08:19:29,465 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:19:29,467 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:19:29,526 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:19:32,534 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:19:33,556 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:19:33,556 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:e7abb38f-482c-4bb6-b9bf-8238b5463dd3 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:19:34,536 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-44) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"1b1a4096-7e66-4faf-b33e-f49d669537ec"} >2014-08-11 08:19:41,000 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:22:34,572 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:e7abb38f-482c-4bb6-b9bf-8238b5463dd3 > > >2014-08-11 08:22:34,573 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:cd35f8e5-446b-442f-89ed-3e28fbe9adfa > > >2014-08-11 08:22:34,572 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@5d4b33b9 >2014-08-11 08:22:34,574 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) HostName = h2 >2014-08-11 08:22:34,575 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:22:34,575 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:22:34,623 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:22:37,633 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:22:38,654 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:22:38,654 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:7f522faf-6c25-45cc-b10c-b7acf34ac7e2 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:22:39,635 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-7) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"766366a8-57c2-4636-be98-e875383a0d41"} >2014-08-11 08:22:41,451 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:25:39,653 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:7f522faf-6c25-45cc-b10c-b7acf34ac7e2 > > >2014-08-11 08:25:39,654 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@38d28410 >2014-08-11 08:25:39,655 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:36c80a01-92d0-4bc2-9111-2b97724a1152 > > >2014-08-11 08:25:39,656 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) HostName = h2 >2014-08-11 08:25:39,658 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:25:39,659 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:25:39,691 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:25:42,699 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:25:43,723 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:25:43,754 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:414896a4-fe7d-4179-b091-678a262bc86f >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:25:44,701 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-60) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"414d0ef5-bfef-4c20-bac2-29ae64c87e0c"} >2014-08-11 08:25:51,929 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:28:44,720 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:414896a4-fe7d-4179-b091-678a262bc86f > > >2014-08-11 08:28:44,721 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:6fd3a997-ce44-481d-80a1-aac85533b5c5 > > >2014-08-11 08:28:44,721 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@1f5debe3 >2014-08-11 08:28:44,722 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) HostName = h2 >2014-08-11 08:28:44,722 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:28:44,723 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:28:44,755 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:28:47,764 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:28:48,784 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:28:48,784 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:f390d855-2c26-40ec-bb2c-c282aff8bd42 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:28:49,768 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-22) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"6d6d2bf1-e266-4f1f-8400-b05ad00383de"} >2014-08-11 08:28:52,378 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:31:49,784 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:f390d855-2c26-40ec-bb2c-c282aff8bd42 > > >2014-08-11 08:31:49,784 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@4751659b >2014-08-11 08:31:49,785 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:922f0b69-f194-44b8-b8f8-ceadb83199c2 > > >2014-08-11 08:31:49,785 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) HostName = h2 >2014-08-11 08:31:49,786 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:31:49,787 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:31:49,844 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:31:52,878 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:31:53,894 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:31:53,894 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:22f45d32-51f5-4217-81b4-40bb1bc627e1 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:31:54,880 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-47) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"c987334b-ca7f-44d3-af67-efcb8a5b722f"} >2014-08-11 08:32:03,253 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:34:54,895 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:22f45d32-51f5-4217-81b4-40bb1bc627e1 > > >2014-08-11 08:34:54,895 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@6c17a169 >2014-08-11 08:34:54,895 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:c33f37f9-2c20-4265-9be0-1aa3834cc4a7 > > >2014-08-11 08:34:54,896 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) HostName = h2 >2014-08-11 08:34:54,897 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:34:54,897 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:34:54,903 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:34:57,910 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:35:40,974 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:35:40,974 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:88a59665-9b57-46bd-9fb9-87a5a0ad86f7 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:35:41,951 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-52) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"25b09b1c-35a1-4c62-9a45-7a90f0ed9b14"} >2014-08-11 08:35:44,166 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:38:41,992 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:88a59665-9b57-46bd-9fb9-87a5a0ad86f7 > > >2014-08-11 08:38:41,992 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@2c37ecc0 >2014-08-11 08:38:41,992 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:071ae3d6-17b3-4992-b791-2a65cecf77db > > >2014-08-11 08:38:41,993 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) HostName = h2 >2014-08-11 08:38:41,993 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:38:41,994 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:38:42,033 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:38:45,042 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:38:51,063 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:38:51,063 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:d38fa0c8-ee30-4afc-9fe2-9434f212c749 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:38:52,049 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-2) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"841db1db-ffd0-4b3f-9ed3-4e734c93ebd9"} >2014-08-11 08:38:54,647 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:41:52,064 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:d38fa0c8-ee30-4afc-9fe2-9434f212c749 > > >2014-08-11 08:41:52,065 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@47b4450b >2014-08-11 08:41:52,066 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:03d15266-061f-4e83-a5f7-1fc18e11fc7b > > >2014-08-11 08:41:52,067 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) HostName = h2 >2014-08-11 08:41:52,068 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:41:52,069 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:41:52,147 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:41:55,161 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:41:56,188 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:41:56,189 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:c721864d-52aa-4c20-96f0-a150c6f6aefb >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:41:57,163 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-4) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"9b545fbb-caf8-4941-a6b5-ca025016f415"} >2014-08-11 08:42:05,138 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:44:57,182 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:c721864d-52aa-4c20-96f0-a150c6f6aefb > > >2014-08-11 08:44:57,182 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@5c6532a3 >2014-08-11 08:44:57,183 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:caa96a02-f657-453f-afe3-c1f82b3f9076 > > >2014-08-11 08:44:57,183 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) HostName = h2 >2014-08-11 08:44:57,185 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:44:57,186 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:44:57,221 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:45:00,234 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:45:01,259 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:45:01,260 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:2dba7e72-449d-4b9f-8003-f676bfbaeb00 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:45:02,237 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-42) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"006333b4-b0c0-4f67-a3a1-2e860c9d93d8"} >2014-08-11 08:45:05,577 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:48:02,255 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:2dba7e72-449d-4b9f-8003-f676bfbaeb00 > > >2014-08-11 08:48:02,256 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [560649b3] Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@3001814b >2014-08-11 08:48:02,256 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:3ef7fa7c-e7f7-47fb-9d3d-48dfce6e8cfa > > >2014-08-11 08:48:02,258 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [560649b3] HostName = h2 >2014-08-11 08:48:02,259 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [560649b3] Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:48:02,261 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [560649b3] Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:48:02,324 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [560649b3] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:48:05,340 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:48:06,368 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:48:06,370 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:fe65b7b2-de84-420a-8333-7e5ab9ecafc7 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:48:07,343 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-91) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"13d8f53f-ed84-4269-a578-2ab6b837218f"} >2014-08-11 08:48:16,063 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:51:07,408 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:fe65b7b2-de84-420a-8333-7e5ab9ecafc7 > > >2014-08-11 08:51:07,408 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@4160b5ca >2014-08-11 08:51:07,408 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:3049d8b2-5284-48ef-8361-e394cfe15a03 > > >2014-08-11 08:51:07,409 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) HostName = h2 >2014-08-11 08:51:07,409 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:51:07,410 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:51:07,445 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:51:10,454 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:51:47,504 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:51:47,504 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:77e3898a-5689-4ebf-aa3f-992bc25f9f2e >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:51:48,490 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-67) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"4c34d78d-f33e-453d-a9b8-0b4797afacdb"} >2014-08-11 08:51:56,620 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:54:48,509 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:77e3898a-5689-4ebf-aa3f-992bc25f9f2e > > >2014-08-11 08:54:48,509 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@fabfa6e >2014-08-11 08:54:48,510 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:2d8ed125-71d9-45fb-b8dd-0d0388bf58bb > > >2014-08-11 08:54:48,511 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) HostName = h2 >2014-08-11 08:54:48,513 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:54:48,515 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:54:48,567 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:54:51,579 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:55:05,615 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:55:05,616 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:1c32b541-8948-4325-a58a-ae631f4a5c22 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:55:06,594 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-17) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"155a861b-ec6e-4526-8997-f739d9593bde"} >2014-08-11 08:55:07,599 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 08:58:06,610 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:1c32b541-8948-4325-a58a-ae631f4a5c22 > > >2014-08-11 08:58:06,611 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@386a1e32 >2014-08-11 08:58:06,611 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:4793fbef-9b4c-4364-842e-0ada3d40b2d5 > > >2014-08-11 08:58:06,611 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) HostName = h2 >2014-08-11 08:58:06,612 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 08:58:06,613 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 08:58:06,668 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 08:58:09,676 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 08:58:24,703 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 08:58:24,703 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:6071b09d-022c-4249-adc7-c2da916c6b97 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 08:58:25,690 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-9) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"455dd651-823b-4fc9-9683-83b934dbc6e0"} >2014-08-11 08:58:28,099 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:01:25,706 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:6071b09d-022c-4249-adc7-c2da916c6b97 > > >2014-08-11 09:01:25,706 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@2a6f5d33 >2014-08-11 09:01:25,706 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:b27338b4-9eee-4220-b457-1dcc6ce615f0 > > >2014-08-11 09:01:25,707 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) HostName = h2 >2014-08-11 09:01:25,707 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:01:25,708 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:01:25,757 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:01:28,767 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:01:46,799 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:01:46,800 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:af3e9147-63d5-45e8-9496-e0f4f3f35460 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:01:47,784 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-93) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"0ceb6924-ab20-4f0c-8ea1-0cd5de1669a5"} >2014-08-11 09:01:48,612 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:03:50,736 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-63) Attempting to update VMs/Templates Ovf. >2014-08-11 09:03:50,859 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-66) Principal admin::internal synchronized >2014-08-11 09:04:47,819 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:af3e9147-63d5-45e8-9496-e0f4f3f35460 > > >2014-08-11 09:04:47,819 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@77f7e16f >2014-08-11 09:04:47,820 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) HostName = h2 >2014-08-11 09:04:47,821 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:04:47,822 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:04:47,820 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:05637488-026e-43cf-934a-2bdd39430758 > > >2014-08-11 09:04:48,146 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:04:51,155 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:04:52,172 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:04:52,173 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:f06d3b41-2363-42bd-bb5f-c3e0cfd5e356 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:04:53,157 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-23) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"11a8610b-291f-4e83-9aa6-cf9383a65b76"} >2014-08-11 09:04:59,097 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:07:53,170 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:f06d3b41-2363-42bd-bb5f-c3e0cfd5e356 > > >2014-08-11 09:07:53,171 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@3ee2b279 >2014-08-11 09:07:53,171 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:8b186655-bc13-4398-a338-1affd574f23d > > >2014-08-11 09:07:53,171 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) HostName = h2 >2014-08-11 09:07:53,172 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:07:53,173 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:07:53,194 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:07:56,202 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:07:58,219 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:07:58,219 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:72fe40f4-6d98-4c21-b54e-5fb20c345f78 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:07:59,205 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-80) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"b2535a73-abe8-45c9-833c-e3b90c6e1e4f"} >2014-08-11 09:07:59,551 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:10:59,215 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:72fe40f4-6d98-4c21-b54e-5fb20c345f78 > > >2014-08-11 09:10:59,215 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@137cfb14 >2014-08-11 09:10:59,216 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:3bd69002-4ce9-4f3e-bd92-2eb7698b70cb > > >2014-08-11 09:10:59,216 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) HostName = h2 >2014-08-11 09:10:59,217 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:10:59,218 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:10:59,272 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:11:02,281 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:11:04,298 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:11:04,298 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:00214640-ec75-4b75-821c-1fd7513ceefa >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:11:05,283 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-46) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"c16b4c5d-5ef9-489b-8cf4-7e3988ea020b"} >2014-08-11 09:11:10,039 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:14:05,298 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:00214640-ec75-4b75-821c-1fd7513ceefa > > >2014-08-11 09:14:05,298 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:8a180258-aca4-4c27-ad77-6a77ad9c4ec2 > > >2014-08-11 09:14:05,299 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@68412352 >2014-08-11 09:14:05,300 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) HostName = h2 >2014-08-11 09:14:05,300 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:14:05,301 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:14:05,359 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-46) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:14:08,370 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:14:09,387 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:14:09,388 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:7343958d-754a-4863-b43d-02abbcd0dae5 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:14:10,378 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-93) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"eedb5a00-0313-4300-a43c-6a65001a3a84"} >2014-08-11 09:14:10,487 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:17:10,392 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:7343958d-754a-4863-b43d-02abbcd0dae5 > > >2014-08-11 09:17:10,392 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@2d0f64bf >2014-08-11 09:17:10,392 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:6a3a267a-87c4-435a-878a-04a0feebd6fc > > >2014-08-11 09:17:10,393 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) HostName = h2 >2014-08-11 09:17:10,394 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:17:10,395 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:17:10,408 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:17:13,419 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:17:14,435 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:17:14,435 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:d1f26593-d161-4075-b29f-c10e68c3a71b >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:17:15,421 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-68) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"53d295e7-f1c2-4bf5-a23a-8b51ad112db3"} >2014-08-11 09:17:20,962 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:20:15,431 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:d1f26593-d161-4075-b29f-c10e68c3a71b > > >2014-08-11 09:20:15,432 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@4d25a11a >2014-08-11 09:20:15,432 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:9f4e9f3d-0e41-42c2-a809-fab35c3bc53d > > >2014-08-11 09:20:15,433 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) HostName = h2 >2014-08-11 09:20:15,434 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:20:15,434 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-68) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:20:15,469 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-68) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:20:18,478 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:20:19,498 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:20:19,498 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:8edf87b3-efaa-4ff6-bbbc-54ffe52b9fdd >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:20:20,480 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-26) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"ebf9a5c9-ea0c-41ae-a834-b61ecf7a4a1c"} >2014-08-11 09:20:21,423 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:23:20,493 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:8edf87b3-efaa-4ff6-bbbc-54ffe52b9fdd > > >2014-08-11 09:23:20,493 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@9d2315d >2014-08-11 09:23:20,494 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:91f56e49-0d36-4928-bcab-5aef342d7dee > > >2014-08-11 09:23:20,494 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) HostName = h2 >2014-08-11 09:23:20,495 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:23:20,496 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:23:20,523 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:23:23,536 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:23:24,562 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:23:24,562 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:e7ac4758-b9e4-47f8-83d6-daf777e1586f >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:23:25,540 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-71) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"43573803-7754-4d27-ae3c-55096753171d"} >2014-08-11 09:23:31,903 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:26:25,554 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:e7ac4758-b9e4-47f8-83d6-daf777e1586f > > >2014-08-11 09:26:25,554 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@50a117ce >2014-08-11 09:26:25,554 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:3fdffe6d-9baa-4c8f-95b9-d36e7453f693 > > >2014-08-11 09:26:25,555 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) HostName = h2 >2014-08-11 09:26:25,556 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:26:25,556 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:26:25,586 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:26:28,596 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:26:29,616 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:26:29,616 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:b4deb10e-64da-491a-a32e-50730ebd345e >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:26:30,598 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-47) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"287096af-de85-4747-a45b-30d1fd869f5c"} >2014-08-11 09:26:32,366 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:29:30,612 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:b4deb10e-64da-491a-a32e-50730ebd345e > > >2014-08-11 09:29:30,612 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@64b66cb7 >2014-08-11 09:29:30,613 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:ee47d3c2-d7b9-4956-ae5f-a842e3fb6be6 > > >2014-08-11 09:29:30,613 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) HostName = h2 >2014-08-11 09:29:30,614 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:29:30,614 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:29:30,625 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:29:33,633 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:29:41,656 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:29:41,657 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:028cb75a-9b58-4c5c-9a56-899a1b9ef7b5 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:29:42,641 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-42) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"0167dac9-2c64-47b1-be3b-47088ffea0b0"} >2014-08-11 09:29:42,844 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:32:42,644 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:028cb75a-9b58-4c5c-9a56-899a1b9ef7b5 > > >2014-08-11 09:32:42,644 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [560649b3] Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@3a352d75 >2014-08-11 09:32:42,644 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:3a9dc6ad-84f5-46bf-a15a-5946736d2f85 > > >2014-08-11 09:32:42,645 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [560649b3] HostName = h2 >2014-08-11 09:32:42,645 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [560649b3] Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:32:42,646 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [560649b3] Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:32:42,697 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [560649b3] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:32:45,705 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:32:46,724 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:32:46,724 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:16639218-3d13-4899-8b1f-4664dea9fdf2 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:32:47,708 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-75) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"86ce8bd3-36a1-4ed1-bd24-0d66d99bb5cd"} >2014-08-11 09:32:53,310 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:35:47,725 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:16639218-3d13-4899-8b1f-4664dea9fdf2 > > >2014-08-11 09:35:47,726 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:339efe1b-90b2-4568-963a-cea9c6e5701d > > >2014-08-11 09:35:47,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@6d7ed90a >2014-08-11 09:35:47,728 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) HostName = h2 >2014-08-11 09:35:47,728 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:35:47,729 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:35:47,744 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:35:50,752 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:35:52,770 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:35:52,770 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:d117b5f3-87f8-4627-ae58-d76d405187ac >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:35:53,756 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-93) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"68932de9-a299-4937-b434-439e6a10b5ec"} >2014-08-11 09:35:53,842 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:38:53,761 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:d117b5f3-87f8-4627-ae58-d76d405187ac > > >2014-08-11 09:38:53,761 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@27267a51 >2014-08-11 09:38:53,761 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:4ae2472a-0df5-46b6-a3d5-8412e717cb2a > > >2014-08-11 09:38:53,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) HostName = h2 >2014-08-11 09:38:53,762 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:38:53,763 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:38:53,807 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:38:56,816 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:38:57,840 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:38:57,840 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:4e876fba-5128-440e-a94c-0b6cb0e76ef0 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:38:58,818 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-67) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"f4e3cd7d-3d11-4900-bb42-726cf5ea3fda"} >2014-08-11 09:39:04,800 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:41:58,824 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:4e876fba-5128-440e-a94c-0b6cb0e76ef0 > > >2014-08-11 09:41:58,824 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@69765b3d >2014-08-11 09:41:58,825 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:d645b4ae-98c7-4e52-8722-7e76a35115d9 > > >2014-08-11 09:41:58,825 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) HostName = h2 >2014-08-11 09:41:58,826 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:41:58,827 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:41:58,839 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:42:01,848 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:42:02,871 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:42:02,872 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:f7652d20-ffb7-4446-9f9f-276a830fae83 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:42:03,850 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-70) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"27b79005-be03-4266-8ae3-58eb1910d051"} >2014-08-11 09:42:05,256 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:45:03,859 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: UNSUBSCRIBE >id:f7652d20-ffb7-4446-9f9f-276a830fae83 > > >2014-08-11 09:45:03,859 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@424d47b5 >2014-08-11 09:45:03,860 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (Response tracker for galactica) Message sent: DISCONNECT >receipt:3c3d3490-4351-489b-9ff8-c3f94e522ca2 > > >2014-08-11 09:45:03,860 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) HostName = h2 >2014-08-11 09:45:03,861 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-08-11 09:45:03,862 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 09:45:03,883 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-08-11 09:45:06,893 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 09:45:07,909 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: CONNECT >heart-beat:0,9000 >accept-version:1.2 > > >2014-08-11 09:45:07,909 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (SSL Stomp Reactor) Message sent: SUBSCRIBE >id:cf90fd49-1459-4982-adcc-3be6061529f6 >ack:client >destination:/queue/_local/vdsm/reponses > > >2014-08-11 09:45:08,895 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient] (DefaultQuartzScheduler_Worker-79) Message sent: SEND >destination:/queue/_local/vdsm/requests >content-length:105 > >{"jsonrpc":"2.0","method":"Host.getCapabilities","params":{},"id":"ad383ce2-62ed-412d-a9d8-bf97ace15697"} >2014-08-11 09:45:15,732 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:225) [rt.jar:1.7.0_25] > at sun.nio.ch.IOUtil.read(IOUtil.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:375) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:78) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:204) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:145) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:116) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:82) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 09:46:27,761 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-1) Running command: LoginAdminUserCommand internal: false. >2014-08-11 09:46:27,802 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-11 09:46:27,945 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-5) Running command: LoginUserCommand internal: false. >2014-08-11 09:46:50,934 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (http--0.0.0.0-8080-1) [59fe3a97] Lock Acquired to object EngineLock [exclusiveLocks= key: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 value: VDS_FENCE >, sharedLocks= ] >2014-08-11 09:46:50,934 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (org.ovirt.thread.pool-8-thread-18) [59fe3a97] Running command: FenceVdsManualyCommand internal: false. Entities affected : ID: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-11 09:46:50,941 INFO [org.ovirt.engine.core.bll.ClearNonResponsiveVdsVmsCommand] (org.ovirt.thread.pool-8-thread-18) [8afa873] Running command: ClearNonResponsiveVdsVmsCommand internal: true. Entities affected : ID: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 Type: VDS >2014-08-11 09:46:50,948 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (org.ovirt.thread.pool-8-thread-18) [8afa873] START, UpdateVdsVMsClearedVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6), log id: 521ef37d >2014-08-11 11:15:38,788 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-11 11:15:38,790 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-11 11:15:38,792 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-11 11:15:38,795 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-11 11:15:38,796 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-11 11:15:38,803 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-11 11:15:38,804 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-11 11:15:38,805 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-11 11:15:38,805 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-11 11:15:38,808 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-11 11:15:38,809 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-11 11:15:38,810 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-11 11:15:38,811 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-11 11:15:38,836 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-11 11:15:38,838 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-11 11:15:38,847 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-11 11:15:38,848 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-11 11:15:38,849 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-11 11:15:38,850 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-11 11:15:38,851 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-11 11:15:38,852 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-11 11:15:38,853 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-11 11:15:38,853 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_USER" is "engine". >2014-08-11 11:15:38,854 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-11 11:15:38,855 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-11 11:15:38,855 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-11 11:15:38,857 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-11 11:15:38,857 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-11 11:15:38,858 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-11 11:15:38,859 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-11 11:15:38,866 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-11 11:15:38,869 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-11 11:15:38,870 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-11 11:15:38,872 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-11 11:15:38,874 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-11 11:15:38,875 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-11 11:15:38,884 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-11 11:15:38,885 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-11 11:15:38,886 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-11 11:15:38,886 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-11 11:15:38,887 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-11 11:15:38,888 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-11 11:15:38,889 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-11 11:15:38,890 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-11 11:15:38,890 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-11 11:15:38,891 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-11 11:15:38,892 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-11 11:15:38,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-11 11:15:38,899 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-11 11:15:38,902 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-11 11:15:38,904 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-11 11:15:38,905 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-11 11:15:38,905 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-11 11:15:38,906 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-11 11:15:38,907 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-11 11:15:38,908 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-11 11:15:38,908 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-11 11:15:38,911 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-11 11:15:38,913 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-11 11:15:38,914 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-11 11:15:38,915 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-11 11:15:38,916 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-11 11:15:38,916 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-11 11:15:38,917 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_USER" is "ovirt". >2014-08-11 11:15:38,918 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-11 11:15:38,918 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-11 11:15:38,919 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-11 11:15:38,920 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-11 11:15:38,920 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-11 11:15:38,921 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-11 11:15:39,311 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Start initializing Backend >2014-08-11 11:15:39,442 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-7) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 11:15:39,557 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-7) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 11:15:39,599 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-7) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-11 11:15:39,613 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-7) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-11 11:15:39,748 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Running ovirt-engine 3.5.0.0 >2014-08-11 11:15:39,749 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-7) Start initializing dictionaries >2014-08-11 11:15:39,751 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-7) Finished initializing dictionaries >2014-08-11 11:15:39,752 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Start initializing AuditLogCleanupManager >2014-08-11 11:15:39,752 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-11 11:15:39,767 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Finished initializing AuditLogCleanupManager >2014-08-11 11:15:39,767 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Start initializing CommandEntityCleanupManager >2014-08-11 11:15:39,768 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-11 11:15:39,769 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Finished initializing CommandEntityCleanupManager >2014-08-11 11:15:39,770 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-7) Start initializing TagsDirector >2014-08-11 11:15:39,785 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-7) Tag root added to tree >2014-08-11 11:15:39,799 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-7) Finished initializing TagsDirector >2014-08-11 11:15:39,800 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-7) Start initializing IsoDomainListSyncronizer >2014-08-11 11:15:39,806 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-7) Finished initializing IsoDomainListSyncronizer >2014-08-11 11:15:39,895 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Completed initializing handlers >2014-08-11 11:15:39,917 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Start initializing ErrorTranslatorImpl >2014-08-11 11:15:39,928 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-11 11:15:39,930 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Finished initializing ErrorTranslatorImpl >2014-08-11 11:15:39,930 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Start initializing ErrorTranslatorImpl >2014-08-11 11:15:39,931 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Finished initializing ErrorTranslatorImpl >2014-08-11 11:15:39,932 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Mark incomplete jobs as UNKNOWN >2014-08-11 11:15:39,975 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-7) Start initializing JobRepositoryCleanupManager >2014-08-11 11:15:39,976 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-7) Finished initializing JobRepositoryCleanupManager >2014-08-11 11:15:39,976 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-7) Start initializing AutoRecoveryManager >2014-08-11 11:15:39,977 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-7) Finished initializing AutoRecoveryManager >2014-08-11 11:15:39,979 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-7) Start initializing ExecutionMessageDirector >2014-08-11 11:15:39,982 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-7) Finished initializing ExecutionMessageDirector >2014-08-11 11:15:40,034 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-7) Start of enabled extensions list >2014-08-11 11:15:40,034 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-7) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-11 11:15:40,035 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-7) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-11 11:15:40,035 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-7) End of enabled extensions list >2014-08-11 11:15:40,098 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-7) Start initializing DbUserCacheManager >2014-08-11 11:15:40,100 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-7) Finished initializing DbUserCacheManager >2014-08-11 11:15:40,119 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-7) Initialization of AsyncTaskManager completed successfully. >2014-08-11 11:15:40,121 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-7) Start initializing ResourceManager >2014-08-11 11:15:40,154 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-11 11:15:40,207 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-7) Entered VdsManager constructor >2014-08-11 11:15:40,243 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-7) Initialize vdsBroker (galactica,54,321) >2014-08-11 11:15:40,323 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-7) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-11 11:15:40,324 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-7) Entered VdsManager constructor >2014-08-11 11:15:40,338 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-7) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-11 11:15:40,420 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-7) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-11 11:15:40,452 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-7) Finished initializing ResourceManager >2014-08-11 11:15:40,456 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-7) Initialization of OvfDataUpdater completed successfully. >2014-08-11 11:15:40,457 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) Start scheduling to enable vds load balancer >2014-08-11 11:15:40,461 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) Finished scheduling to enable vds load balancer >2014-08-11 11:15:40,462 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) Start HA Reservation check >2014-08-11 11:15:40,463 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) Finished HA Reservation check >2014-08-11 11:15:40,474 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-7) Init VM custom properties utilities >2014-08-11 11:15:40,475 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-7) Init device custom properties utilities >2014-08-11 11:15:40,479 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) Initializing Scheduling manager >2014-08-11 11:15:40,527 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-11 11:15:40,528 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-11 11:15:40,528 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-11 11:15:40,531 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) External scheduler disabled, discovery skipped >2014-08-11 11:15:40,532 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-7) Initialized Scheduling manager >2014-08-11 11:15:40,532 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-7) Initializing DWH Heart Beat >2014-08-11 11:15:40,533 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-7) DWH Heart Beat initialized >2014-08-11 11:15:43,385 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:15:44,034 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: d9bdcc2 >2014-08-11 11:15:44,046 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GetHardwareInfoVDSCommand, log id: d9bdcc2 >2014-08-11 11:15:44,052 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-11 11:15:44,061 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: UNASSIGNED not exist in string table >2014-08-11 11:15:44,064 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-11 11:15:44,064 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-11 11:15:44,067 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-11 11:15:44,068 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-11 11:15:44,068 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-11 11:15:44,069 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-11 11:15:44,069 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-11 11:15:44,077 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-11 11:15:44,078 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-11 11:15:44,079 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-11 11:15:44,079 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: DWH_STOPPED not exist in string table >2014-08-11 11:15:44,080 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: DWH_STARTED not exist in string table >2014-08-11 11:15:44,081 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: DWH_ERROR not exist in string table >2014-08-11 11:15:44,081 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-11 11:15:44,082 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-11 11:15:44,083 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-11 11:15:44,084 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-11 11:15:44,324 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-11 11:15:44,325 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is running with disabled SELinux. >2014-08-11 11:15:44,592 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: h1 >2014-08-11 11:15:44,742 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-7) [1c92cc2f] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-11 11:15:50,479 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-21) [142ad1c] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-11 11:15:50,505 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-21) [142ad1c] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-11 11:15:50,514 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-21) [142ad1c] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-11 11:15:50,843 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [142ad1c] Correlation ID: 142ad1c, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-11 11:15:50,985 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-21) [142ad1c] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-11 11:15:51,135 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-21) [142ad1c] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-11 11:15:51,165 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [142ad1c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-11 11:15:51,186 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-15) [142ad1c] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 561e6ae7 >2014-08-11 11:15:51,281 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [142ad1c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 52 GB of free space >2014-08-11 11:15:51,306 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [142ad1c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 52 GB of free space >2014-08-11 11:15:51,307 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-15) [142ad1c] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-11 11:15:51,320 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-15) [142ad1c] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 4bf9c513 >2014-08-11 11:15:51,327 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-15) [142ad1c] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 4bf9c513 >2014-08-11 11:15:51,328 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-15) [142ad1c] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 561e6ae7 >2014-08-11 11:15:51,328 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-15) [142ad1c] Discovered no tasks on Storage Pool Default >2014-08-11 11:16:01,467 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:01,471 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:01,481 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-6) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:01,493 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:01,495 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h2 is not responding. It will stay in Connecting state for a grace period of $60 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:16:01,551 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:01,560 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Failed to refresh VDS , vds = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 : h2, VDS Network Error, continuing. >java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:04,581 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:16:07,587 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:07,590 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:07,595 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-51) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:07,604 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:07,606 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Host h2 is not responding. It will stay in Connecting state for a grace period of $60 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:16:07,607 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:07,618 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Failed to refresh VDS , vds = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 : h2, VDS Network Error, continuing. >java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:10,655 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:16:13,661 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:13,664 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:13,666 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-71) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:13,674 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:13,675 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h2 is not responding. It will stay in Connecting state for a grace period of $60 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:16:13,712 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Server failed to respond, vds_id = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds_name = h2, vm_count = 0, spm_status = None, non-responsive_timeout (seconds) = 60, error = java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:13,759 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-71) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) > at sun.nio.ch.Net.connect(Net.java:465) > at sun.nio.ch.Net.connect(Net.java:457) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) >, Custom Event ID: -1, Message: Host h2 is non responsive. >2014-08-11 11:16:13,766 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-8-thread-41) ResourceManager::vdsNotResponding entered for Host eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, galactica >2014-08-11 11:16:13,784 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:13,842 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-41) [6957e289] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 Type: VDS >2014-08-11 11:16:13,846 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-41) [6957e289] Opening SSH Soft Fencing session on host galactica >2014-08-11 11:16:16,668 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-41) [6957e289] SSH Soft Fencing command failed on host galactica: Failed to get the session. >Stdout: >Stderr: >Stacktrace: org.apache.sshd.common.RuntimeSshException: Failed to get the session.: org.apache.sshd.common.RuntimeSshException: Failed to get the session. > at org.apache.sshd.client.future.DefaultConnectFuture.getSession(DefaultConnectFuture.java:43) > at org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:294) [uutils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:87) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:53) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1163) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1290) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1910) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1311) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view6.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$4.run(VdsEventListener.java:240) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishConnect(UnixAsynchronousSocketChannelImpl.java:252) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:213) [rt.jar:1.7.0_25] > at sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:293) [rt.jar:1.7.0_25] > ... 1 more > >2014-08-11 11:16:16,769 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-41) [6757481e] Lock Acquired to object EngineLock [exclusiveLocks= key: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 value: VDS_FENCE >, sharedLocks= ] >2014-08-11 11:16:16,770 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-41) [6757481e] Failed to run Fence script on vds:h2. >2014-08-11 11:16:16,775 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-41) [6757481e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h2 became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-08-11 11:16:16,777 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-41) [6757481e] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-08-11 11:16:16,777 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-41) [6757481e] Lock freed to object EngineLock [exclusiveLocks= key: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 value: VDS_FENCE >, sharedLocks= ] >2014-08-11 11:16:16,803 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:16:19,673 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:19,675 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:19,677 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-83) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:19,684 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:19,684 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:16:19,730 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:22,778 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:16:25,783 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:25,785 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:25,787 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-100) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:25,795 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:25,796 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:16:25,834 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:28,845 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:16:31,851 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:31,854 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:31,857 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-12) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:31,879 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:31,880 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:16:32,012 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:35,034 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:16:38,039 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:38,042 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:38,044 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-29) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:38,056 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:38,057 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-29) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:16:38,108 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:39,910 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-1) Running command: LoginAdminUserCommand internal: false. >2014-08-11 11:16:39,938 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-11 11:16:41,169 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:16:43,780 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-1) Running command: LoginUserCommand internal: false. >2014-08-11 11:16:44,175 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:44,178 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:44,181 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-54) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:44,193 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:44,195 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:16:44,321 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:47,369 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:16:50,375 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:50,379 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:50,382 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-66) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:50,388 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:50,389 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:16:50,415 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:53,431 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:16:56,439 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:56,442 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:56,445 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-82) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:56,454 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:16:56,455 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:16:56,477 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:16:59,492 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:17:02,499 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:02,502 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:02,504 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-93) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:02,511 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:17:02,511 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:17:02,523 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:05,539 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:17:08,545 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:08,547 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:08,549 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-13) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:08,556 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:17:08,556 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:17:08,610 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:11,591 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [13d3c1a3] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-11 11:17:11,625 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:17:11,800 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] Running command: HibernateVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 11:17:11,836 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@b78ec69a), log id: eb2b11 >2014-08-11 11:17:11,838 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] FINISH, UpdateVmDynamicDataVDSCommand, log id: eb2b11 >2014-08-11 11:17:11,851 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = ee28320c-7b27-4dfe-a4dc-a00f523ed516, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = a2e552b6-5e95-450b-aebf-1577a41d70e1, newImageDescription = ), log id: 7b6464e4 >2014-08-11 11:17:11,858 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 11:17:12,161 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] FINISH, CreateImageVDSCommand, return: a2e552b6-5e95-450b-aebf-1577a41d70e1, log id: 7b6464e4 >2014-08-11 11:17:12,192 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command abd587b6-9969-4d8f-a11d-406436f5b925 >2014-08-11 11:17:12,195 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] CommandMultiAsyncTasks::AttachTask: Attaching task 4ce36ca0-0d23-420c-9493-6f59ea013009 to command abd587b6-9969-4d8f-a11d-406436f5b925. >2014-08-11 11:17:12,220 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] Adding task 4ce36ca0-0d23-420c-9493-6f59ea013009 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 11:17:12,239 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = d98ca030-8e70-4ea9-88ea-195101cf42de, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = 03c164c3-297b-41a1-a95e-29d6560b94b5, newImageDescription = ), log id: 31bae4ab >2014-08-11 11:17:12,241 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 11:17:12,721 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] FINISH, CreateImageVDSCommand, return: 03c164c3-297b-41a1-a95e-29d6560b94b5, log id: 31bae4ab >2014-08-11 11:17:12,787 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] CommandMultiAsyncTasks::AttachTask: Attaching task e13035de-d8be-47e0-a7af-3ac25a818c4f to command abd587b6-9969-4d8f-a11d-406436f5b925. >2014-08-11 11:17:12,802 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] Adding task e13035de-d8be-47e0-a7af-3ac25a818c4f (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 11:17:12,813 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@4b3d885b), log id: 4712bd5e >2014-08-11 11:17:12,827 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] FINISH, UpdateVmDynamicDataVDSCommand, log id: 4712bd5e >2014-08-11 11:17:12,893 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] Correlation ID: 13d3c1a3, Job ID: 6b8b5505-20fc-47d3-83a5-1bc51d5c72e2, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm3 was initiated by User admin (Host: h1). >2014-08-11 11:17:12,894 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] BaseAsyncTask::startPollingTask: Starting to poll task 4ce36ca0-0d23-420c-9493-6f59ea013009. >2014-08-11 11:17:12,895 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] BaseAsyncTask::startPollingTask: Starting to poll task e13035de-d8be-47e0-a7af-3ac25a818c4f. >2014-08-11 11:17:12,896 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-13) [13d3c1a3] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-11 11:17:14,631 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:14,633 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:14,635 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-32) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:14,642 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:17:14,642 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:17:14,715 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:17,516 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from PreparingForHibernate --> Up >2014-08-11 11:17:17,726 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:17:20,108 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-6) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 11:17:20,118 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-6) SPMAsyncTask::PollTask: Polling task e13035de-d8be-47e0-a7af-3ac25a818c4f (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 11:17:20,173 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-6) BaseAsyncTask::onTaskEndSuccess: Task e13035de-d8be-47e0-a7af-3ac25a818c4f (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 11:17:20,174 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-6) Task with DB Task ID 562399aa-8d72-4821-b48c-e29db4bfe210 and VDSM Task ID 4ce36ca0-0d23-420c-9493-6f59ea013009 is in state Polling. End action for command abd587b6-9969-4d8f-a11d-406436f5b925 will proceed when all the entitys tasks are completed. >2014-08-11 11:17:20,175 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-6) SPMAsyncTask::PollTask: Polling task 4ce36ca0-0d23-420c-9493-6f59ea013009 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 11:17:20,190 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-6) BaseAsyncTask::onTaskEndSuccess: Task 4ce36ca0-0d23-420c-9493-6f59ea013009 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 11:17:20,191 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-6) CommandAsyncTask::endActionIfNecessary: All tasks of command abd587b6-9969-4d8f-a11d-406436f5b925 has ended -> executing endAction >2014-08-11 11:17:20,192 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-6) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: abd587b6-9969-4d8f-a11d-406436f5b925): calling endAction . >2014-08-11 11:17:20,194 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-28) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-11 11:17:20,281 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-28) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-11 11:17:20,284 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-28) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,ee28320c-7b27-4dfe-a4dc-a00f523ed516,a2e552b6-5e95-450b-aebf-1577a41d70e1,d98ca030-8e70-4ea9-88ea-195101cf42de,03c164c3-297b-41a1-a95e-29d6560b94b5), log id: 3235caec >2014-08-11 11:17:20,286 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-28) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,ee28320c-7b27-4dfe-a4dc-a00f523ed516,a2e552b6-5e95-450b-aebf-1577a41d70e1,d98ca030-8e70-4ea9-88ea-195101cf42de,03c164c3-297b-41a1-a95e-29d6560b94b5), log id: 5378123f >2014-08-11 11:17:20,294 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-28) FINISH, HibernateBrokerVDSCommand, log id: 5378123f >2014-08-11 11:17:20,314 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-28) FINISH, HibernateVDSCommand, log id: 3235caec >2014-08-11 11:17:20,318 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-28) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-11 11:17:20,319 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-28) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-11 11:17:20,331 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-28) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 4ce36ca0-0d23-420c-9493-6f59ea013009 >2014-08-11 11:17:20,332 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-28) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 4ce36ca0-0d23-420c-9493-6f59ea013009), log id: 762b29ff >2014-08-11 11:17:20,334 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-28) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=4ce36ca0-0d23-420c-9493-6f59ea013009), log id: 626390b7 >2014-08-11 11:17:20,453 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-28) FINISH, HSMClearTaskVDSCommand, log id: 626390b7 >2014-08-11 11:17:20,453 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-28) FINISH, SPMClearTaskVDSCommand, log id: 762b29ff >2014-08-11 11:17:20,472 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-28) BaseAsyncTask::removeTaskFromDB: Removed task 4ce36ca0-0d23-420c-9493-6f59ea013009 from DataBase >2014-08-11 11:17:20,472 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-28) SPMAsyncTask::ClearAsyncTask: Attempting to clear task e13035de-d8be-47e0-a7af-3ac25a818c4f >2014-08-11 11:17:20,472 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-28) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = e13035de-d8be-47e0-a7af-3ac25a818c4f), log id: 36ef753a >2014-08-11 11:17:20,474 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-28) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=e13035de-d8be-47e0-a7af-3ac25a818c4f), log id: 22f94f03 >2014-08-11 11:17:20,601 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-28) FINISH, HSMClearTaskVDSCommand, log id: 22f94f03 >2014-08-11 11:17:20,602 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-28) FINISH, SPMClearTaskVDSCommand, log id: 36ef753a >2014-08-11 11:17:20,612 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-28) BaseAsyncTask::removeTaskFromDB: Removed task e13035de-d8be-47e0-a7af-3ac25a818c4f from DataBase >2014-08-11 11:17:20,613 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-28) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity abd587b6-9969-4d8f-a11d-406436f5b925 >2014-08-11 11:17:20,664 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from SavingState --> Up >2014-08-11 11:17:20,731 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:20,735 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:20,737 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-42) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:20,744 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:17:20,745 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:17:20,753 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:23,767 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:17:26,773 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:26,775 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:26,779 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-59) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:26,789 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:17:26,790 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:17:26,823 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:29,841 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:17:32,847 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:32,850 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:32,853 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-72) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:32,860 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:17:32,862 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:17:32,870 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:35,902 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:17:38,909 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:38,912 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:38,915 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-82) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:38,930 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:17:38,931 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:17:38,957 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:41,970 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:17:44,975 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:44,980 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:44,983 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-10) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:44,992 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:17:44,992 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:17:45,010 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:48,022 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:17:51,027 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:51,029 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:51,031 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-12) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:51,038 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:17:51,038 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:17:51,089 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:54,108 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:17:57,115 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:57,119 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:57,122 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-41) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:17:57,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:17:57,131 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:17:57,168 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:00,178 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:18:03,183 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:03,186 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:03,189 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-53) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:03,199 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:18:03,201 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:18:03,231 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:06,241 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:18:09,247 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:09,249 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:09,251 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-69) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:09,258 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:18:09,259 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:18:09,303 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:12,312 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:18:15,319 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:15,347 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:15,349 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-88) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:15,356 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:18:15,357 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:18:15,426 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:17,682 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [3e69bf3e] Lock Acquired to object EngineLock [exclusiveLocks= key: 8c01a1b0-4f66-4e56-8a60-49208cb7914f value: VM >, sharedLocks= ] >2014-08-11 11:18:17,812 WARN [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [3e69bf3e] CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_IMAGE_REPOSITORY_NOT_FOUND >2014-08-11 11:18:17,813 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [3e69bf3e] Lock freed to object EngineLock [exclusiveLocks= key: 8c01a1b0-4f66-4e56-8a60-49208cb7914f value: VM >, sharedLocks= ] >2014-08-11 11:18:18,436 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:18:21,443 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:21,445 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:21,447 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-1) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:21,454 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:18:21,454 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:18:21,469 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:24,481 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:18:27,487 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:27,490 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:27,492 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-13) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:27,500 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:18:27,501 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:18:27,532 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:30,542 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:18:33,547 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:33,549 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:33,551 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-35) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:33,558 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:18:33,558 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:18:33,604 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:36,614 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:18:39,621 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:39,624 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:39,626 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-6) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:39,634 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:18:39,635 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:18:39,658 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:40,109 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-55) Setting new tasks map. The map contains now 0 tasks >2014-08-11 11:18:40,110 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-55) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 11:18:42,668 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:18:45,675 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:45,677 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:45,679 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-68) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:45,686 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:18:45,687 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-68) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:18:45,704 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-68) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:48,716 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:18:51,423 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (http--0.0.0.0-8080-2) [1cd17e1b] Lock Acquired to object EngineLock [exclusiveLocks= key: vm value: VM_NAME >, sharedLocks= ] >2014-08-11 11:18:51,468 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (http--0.0.0.0-8080-2) [1cd17e1b] Running command: AddVmFromScratchCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-00000000039f Type: VdsGroupsAction group CREATE_VM with role type USER >2014-08-11 11:18:51,723 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:51,726 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:51,729 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-81) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:51,739 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:18:51,740 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:18:51,750 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:51,759 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [1cd17e1b] Correlation ID: 1cd17e1b, Job ID: 6fd3f616-6638-4b78-80a2-2fef6b565587, Call Stack: null, Custom Event ID: -1, Message: VM vm was created by admin. >2014-08-11 11:18:51,767 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (http--0.0.0.0-8080-2) [1cd17e1b] Lock freed to object EngineLock [exclusiveLocks= key: vm value: VM_NAME >, sharedLocks= ] >2014-08-11 11:18:52,408 INFO [org.ovirt.engine.core.bll.network.vm.AddVmInterfaceCommand] (http--0.0.0.0-8080-2) [4d1ad9d2] Running command: AddVmInterfaceCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group CONFIGURE_VM_NETWORK with role type USER, ID: 0000000e-000e-000e-000e-000000000181 Type: VnicProfileAction group CONFIGURE_VM_NETWORK with role type USER >2014-08-11 11:18:52,543 INFO [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (http--0.0.0.0-8080-2) [5e5337d9] Running command: ActivateDeactivateVmNicCommand internal: true. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VM >2014-08-11 11:18:52,595 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [5e5337d9] Correlation ID: 5e5337d9, Call Stack: null, Custom Event ID: -1, Message: Network Interface nic1 (VirtIO) was plugged to VM vm. (User: admin) >2014-08-11 11:18:52,620 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [5e5337d9] Correlation ID: 4d1ad9d2, Call Stack: null, Custom Event ID: -1, Message: Interface nic1 (VirtIO) was added to VM vm. (User: admin) >2014-08-11 11:18:52,943 INFO [org.ovirt.engine.core.bll.network.vm.ReorderVmNicsCommand] (http--0.0.0.0-8080-2) [45b1a8f2] Running command: ReorderVmNicsCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group CONFIGURE_VM_NETWORK with role type USER >2014-08-11 11:18:54,765 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:18:57,771 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:57,773 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:57,775 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-97) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:18:57,782 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:18:57,783 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:18:57,820 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:00,832 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:19:03,839 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:03,842 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:03,845 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-19) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:03,856 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:19:03,857 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:19:03,900 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:06,925 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:19:09,931 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:09,934 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:09,935 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-27) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:09,942 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:19:09,943 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:19:09,963 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:12,407 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (http--0.0.0.0-8080-4) [5cd9c005] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM_DISK_BOOT >, sharedLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >] >2014-08-11 11:19:12,447 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (http--0.0.0.0-8080-4) [5cd9c005] Running command: AddDiskCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group CONFIGURE_VM_STORAGE with role type USER, ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 11:19:12,563 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (http--0.0.0.0-8080-4) [47b9698d] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-11 11:19:12,613 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (http--0.0.0.0-8080-4) [47b9698d] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM_DISK_BOOT >, sharedLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >] >2014-08-11 11:19:12,614 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (http--0.0.0.0-8080-4) [47b9698d] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 99e5cda5-262b-4469-b336-04e3f0cf7c38, imageSizeInBytes = 10737418240, volumeFormat = RAW, newImageId = 3f8b5bbe-741b-4a4f-9397-84123d530772, newImageDescription = ), log id: f7589b3 >2014-08-11 11:19:12,616 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (http--0.0.0.0-8080-4) [47b9698d] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 11:19:12,976 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:19:13,018 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (http--0.0.0.0-8080-4) [47b9698d] FINISH, CreateImageVDSCommand, return: 3f8b5bbe-741b-4a4f-9397-84123d530772, log id: f7589b3 >2014-08-11 11:19:13,206 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (http--0.0.0.0-8080-4) [47b9698d] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command d0644ea3-7d4f-4d09-8dee-aea6c9856c41 >2014-08-11 11:19:13,207 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (http--0.0.0.0-8080-4) [47b9698d] CommandMultiAsyncTasks::AttachTask: Attaching task a4e55539-6f2a-4cf4-8c09-1a8f3fe19ab5 to command d0644ea3-7d4f-4d09-8dee-aea6c9856c41. >2014-08-11 11:19:13,234 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (http--0.0.0.0-8080-4) [47b9698d] Adding task a4e55539-6f2a-4cf4-8c09-1a8f3fe19ab5 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 11:19:13,399 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [47b9698d] Correlation ID: 5cd9c005, Job ID: c9a35fbb-5681-4667-b1db-bb48227c9fe3, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of vm_Disk1 was initiated on VM vm by admin. >2014-08-11 11:19:13,400 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (http--0.0.0.0-8080-4) [47b9698d] BaseAsyncTask::startPollingTask: Starting to poll task a4e55539-6f2a-4cf4-8c09-1a8f3fe19ab5. >2014-08-11 11:19:15,983 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:15,985 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:15,988 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-46) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:15,996 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:19:15,997 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:19:16,016 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-46) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:19,027 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:19:20,198 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-53) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-11 11:19:20,205 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-53) SPMAsyncTask::PollTask: Polling task a4e55539-6f2a-4cf4-8c09-1a8f3fe19ab5 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 11:19:20,241 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-53) BaseAsyncTask::onTaskEndSuccess: Task a4e55539-6f2a-4cf4-8c09-1a8f3fe19ab5 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 11:19:20,242 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-53) CommandAsyncTask::endActionIfNecessary: All tasks of command d0644ea3-7d4f-4d09-8dee-aea6c9856c41 has ended -> executing endAction >2014-08-11 11:19:20,242 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-53) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: d0644ea3-7d4f-4d09-8dee-aea6c9856c41): calling endAction . >2014-08-11 11:19:20,244 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddDisk, executionIndex: 0 >2014-08-11 11:19:20,257 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (org.ovirt.thread.pool-8-thread-34) [5cd9c005] Ending command successfully: org.ovirt.engine.core.bll.AddDiskCommand >2014-08-11 11:19:20,290 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-8-thread-34) [47b9698d] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-08-11 11:19:20,328 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-34) [47b9698d] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 99e5cda5-262b-4469-b336-04e3f0cf7c38, imageId = 3f8b5bbe-741b-4a4f-9397-84123d530772), log id: 1c1fc822 >2014-08-11 11:19:20,360 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-34) [47b9698d] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@cd85356a, log id: 1c1fc822 >2014-08-11 11:19:20,398 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-34) Correlation ID: 5cd9c005, Call Stack: null, Custom Event ID: -1, Message: The disk vm_Disk1 was successfully added to VM vm. >2014-08-11 11:19:20,398 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk completed, handling the result. >2014-08-11 11:19:20,399 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk succeeded, clearing tasks. >2014-08-11 11:19:20,406 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) SPMAsyncTask::ClearAsyncTask: Attempting to clear task a4e55539-6f2a-4cf4-8c09-1a8f3fe19ab5 >2014-08-11 11:19:20,407 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = a4e55539-6f2a-4cf4-8c09-1a8f3fe19ab5), log id: 39c8760a >2014-08-11 11:19:20,409 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=a4e55539-6f2a-4cf4-8c09-1a8f3fe19ab5), log id: 246b27ec >2014-08-11 11:19:20,575 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) FINISH, HSMClearTaskVDSCommand, log id: 246b27ec >2014-08-11 11:19:20,576 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) FINISH, SPMClearTaskVDSCommand, log id: 39c8760a >2014-08-11 11:19:20,613 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) BaseAsyncTask::removeTaskFromDB: Removed task a4e55539-6f2a-4cf4-8c09-1a8f3fe19ab5 from DataBase >2014-08-11 11:19:20,614 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity d0644ea3-7d4f-4d09-8dee-aea6c9856c41 >2014-08-11 11:19:22,033 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:22,035 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:22,037 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-58) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:22,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:19:22,046 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:19:22,062 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:24,813 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-4) [1e0e9fe7] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 11:19:24,839 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [1e0e9fe7] START, IsVmDuringInitiatingVDSCommand( vmId = 14c0fa91-1814-438b-8d7a-4e7de1e1da25), log id: 2877b5b6 >2014-08-11 11:19:24,840 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [1e0e9fe7] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 2877b5b6 >2014-08-11 11:19:24,929 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-43) [1e0e9fe7] Running command: RunVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 11:19:25,002 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-43) [1e0e9fe7] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@7f48ab0), log id: 55daf2d4 >2014-08-11 11:19:25,012 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-43) [1e0e9fe7] FINISH, UpdateVmDynamicDataVDSCommand, log id: 55daf2d4 >2014-08-11 11:19:25,022 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-43) [1e0e9fe7] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, vm=VM [vm]), log id: 4a24393b >2014-08-11 11:19:25,073 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:19:25,116 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand] (org.ovirt.thread.pool-8-thread-43) [1e0e9fe7] START, CreateVmFromCloudInitVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, vm=VM [vm]), log id: 36c9277d >2014-08-11 11:19:25,164 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-43) [1e0e9fe7] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{shared=false, iface=ide, index=3, specParams={vmPayload={file={openstack/latest/meta_data.json=ewogICJsYXVuY2hfaW5kZXgiIDogIjAiLAogICJhdmFpbGFiaWxpdHlfem9uZSIgOiAibm92YSIs >CiAgIm5ldHdvcmstaW50ZXJmYWNlcyIgOiAiYXV0b1xuIiwKICAibmV0d29ya19jb25maWciIDog >ewogICAgImNvbnRlbnRfcGF0aCIgOiAiL2NvbnRlbnQvMDAwMCIsCiAgICAicGF0aCIgOiAiL2V0 >Yy9uZXR3b3JrL2ludGVyZmFjZXMiCiAgfSwKICAidXVpZCIgOiAiM2JiYzMxMWYtMmY2OS00YTc3 >LWIwZjAtNTJmNzJiZDIwZjA4IiwKICAibWV0YSIgOiB7CiAgICAiZXNzZW50aWFsIiA6ICJmYWxz >ZSIsCiAgICAicm9sZSIgOiAic2VydmVyIiwKICAgICJkc21vZGUiIDogImxvY2FsIgogIH0KfQ== >, openstack/latest/user_data=I2Nsb3VkLWNvbmZpZwpzc2hfcHdhdXRoOiB0cnVlCmRpc2FibGVfcm9vdDogMApvdXRwdXQ6CiAg >YWxsOiAnPj4gL3Zhci9sb2cvY2xvdWQtaW5pdC1vdXRwdXQubG9nJwpjaHBhc3N3ZDoKICBleHBp >cmU6IGZhbHNlCnJ1bmNtZDoKLSAnc2VkIC1pICcnL15kYXRhc291cmNlX2xpc3Q6IC9kJycgL2V0 >Yy9jbG91ZC9jbG91ZC5jZmc7IGVjaG8gJydkYXRhc291cmNlX2xpc3Q6CiAgWyJOb0Nsb3VkIiwg >IkNvbmZpZ0RyaXZlIl0nJyA+PiAvZXRjL2Nsb3VkL2Nsb3VkLmNmZycK >, openstack/content/0000=YXV0bwo= >}, volId=config-2}}, path=, device=cdrom, type=disk, readonly=true, deviceId=771ac016-6f1a-4974-a5bf-55a2f5c40f2c}, {specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=68e21ff2-4b5a-4b29-8765-eec79f1b15dd}, {shared=false, iface=ide, index=2, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=496deb19-46c7-4b58-94a3-06b7fa34f7ac}, {shared=false, volumeID=3f8b5bbe-741b-4a4f-9397-84123d530772, index=0, iface=virtio, imageID=99e5cda5-262b-4469-b336-04e3f0cf7c38, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, specParams={}, optional=false, propagateErrors=off, device=disk, poolID=00000002-0002-0002-0002-00000000037f, format=raw, type=disk, readonly=false, deviceId=99e5cda5-262b-4469-b336-04e3f0cf7c38}, {nicModel=pv, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:57, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=1e5b18b9-16eb-4b96-83b6-4568b8bed355}, {index=0, model=virtio-scsi, specParams={}, device=scsi, type=controller, deviceId=3959560b-27d1-4acf-acd9-c39f0c4148b2}],vmName=vm,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 11:19:25,188 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand] (org.ovirt.thread.pool-8-thread-43) [1e0e9fe7] FINISH, CreateVmFromCloudInitVDSCommand, log id: 36c9277d >2014-08-11 11:19:25,218 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-43) [1e0e9fe7] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 4a24393b >2014-08-11 11:19:25,219 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-43) [1e0e9fe7] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 11:19:25,235 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-43) [1e0e9fe7] Correlation ID: 1e0e9fe7, Job ID: 97825897-63a5-4d1d-9a56-3358f35528c9, Call Stack: null, Custom Event ID: -1, Message: VM vm was started by admin (Host: h1). >2014-08-11 11:19:28,079 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:28,081 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:28,083 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-76) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:28,089 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:19:28,090 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:19:28,125 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:28,740 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) VM vm 14c0fa91-1814-438b-8d7a-4e7de1e1da25 moved from WaitForLaunch --> PoweringUp >2014-08-11 11:19:28,787 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-84) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 36c3b14d >2014-08-11 11:19:28,799 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-84) FINISH, FullListVdsCommand, return: [{displaySecurePort=5902, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.35.0.105, pauseCode=NOERR, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=23099, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@6e72c4e9, vmName=vm, fileTransferEnable=true, cpuType=SandyBridge}], log id: 36c3b14d >2014-08-11 11:19:28,802 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Received a memballoon Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-08-11 11:19:28,803 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Received a spice Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5902, type=graphics} >2014-08-11 11:19:31,137 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:19:34,143 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:34,145 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:34,147 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-95) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:34,154 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:19:34,154 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:19:34,188 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:37,201 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:19:40,207 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:40,209 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:40,211 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-3) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:40,218 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:19:40,219 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:19:40,234 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:43,243 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:19:46,249 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:46,251 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:46,253 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-28) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:46,259 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:19:46,260 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:19:46,304 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:49,315 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to galactica/10.35.0.106 >2014-08-11 11:19:52,321 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:52,323 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:52,325 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-40) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:76) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:52,334 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, vds=Host[h2,eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-11 11:19:52,335 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host h2 is not responding. It will stay in Connecting state for a grace period of $120 seconds and after that an attempt to fence the host will be issued. >2014-08-11 11:19:52,486 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_25] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_25] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:639) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:57) [vdsm-jsonrpc-java-client.jar:] > >2014-08-11 11:19:54,531 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-8-thread-37) [5eba47c0] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-08-11 11:19:54,619 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-8-thread-37) [5eba47c0] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 Type: VDS >2014-08-11 11:19:54,623 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-37) [5eba47c0] START, SetVdsStatusVDSCommand(HostName = h2, HostId = eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 41f51775 >2014-08-11 11:19:54,636 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-37) [5eba47c0] FINISH, SetVdsStatusVDSCommand, log id: 41f51775 >2014-08-11 11:19:54,653 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-37) [5eba47c0] Correlation ID: 5eba47c0, Job ID: 47e7e90e-e484-46be-b184-eb787a0fc022, Call Stack: null, Custom Event ID: -1, Message: Host h2 was switched to Maintenance mode by admin. >2014-08-11 11:19:55,495 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Initializing Host: h2 >2014-08-11 11:20:27,571 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) VM vm 14c0fa91-1814-438b-8d7a-4e7de1e1da25 moved from PoweringUp --> Up >2014-08-11 11:20:27,601 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) Correlation ID: 1e0e9fe7, Job ID: 97825897-63a5-4d1d-9a56-3358f35528c9, Call Stack: null, Custom Event ID: -1, Message: VM vm started on Host h1 >2014-08-11 11:20:40,113 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-78) Setting new tasks map. The map contains now 0 tasks >2014-08-11 11:20:40,114 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-78) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 11:21:47,212 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [494a81b4] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 11:21:47,311 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-18) [494a81b4] Running command: HibernateVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 11:21:47,378 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-18) [494a81b4] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@fc4764d3), log id: 2eeccaa0 >2014-08-11 11:21:47,380 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-18) [494a81b4] FINISH, UpdateVmDynamicDataVDSCommand, log id: 2eeccaa0 >2014-08-11 11:21:47,392 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-18) [494a81b4] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 181d3718-6f5a-43d4-beac-82a4631d87c5, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = aed9b3d4-8c5f-45d7-a3ee-864769c6eb42, newImageDescription = ), log id: 68a3a538 >2014-08-11 11:21:47,394 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-18) [494a81b4] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 11:21:47,794 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-18) [494a81b4] FINISH, CreateImageVDSCommand, return: aed9b3d4-8c5f-45d7-a3ee-864769c6eb42, log id: 68a3a538 >2014-08-11 11:21:47,939 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-18) [494a81b4] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command f36529fb-4f41-43f2-bfaf-034634ca44b4 >2014-08-11 11:21:47,939 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-18) [494a81b4] CommandMultiAsyncTasks::AttachTask: Attaching task 97d9cc2a-1b59-40fe-a519-529410a665ff to command f36529fb-4f41-43f2-bfaf-034634ca44b4. >2014-08-11 11:21:47,957 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-18) [494a81b4] Adding task 97d9cc2a-1b59-40fe-a519-529410a665ff (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 11:21:47,972 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-18) [494a81b4] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b733c16d-9c36-42ce-93ea-c31d53fa2aaa, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = 2db253d5-b04a-4991-8257-fd81190310ad, newImageDescription = ), log id: 54007270 >2014-08-11 11:21:47,973 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-18) [494a81b4] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 11:21:48,611 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-18) [494a81b4] FINISH, CreateImageVDSCommand, return: 2db253d5-b04a-4991-8257-fd81190310ad, log id: 54007270 >2014-08-11 11:21:48,693 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-18) [494a81b4] CommandMultiAsyncTasks::AttachTask: Attaching task 160c52c5-c817-4098-839d-557ba6a23647 to command f36529fb-4f41-43f2-bfaf-034634ca44b4. >2014-08-11 11:21:48,733 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-18) [494a81b4] Adding task 160c52c5-c817-4098-839d-557ba6a23647 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 11:21:48,743 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-18) [494a81b4] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@45d43a04), log id: 7fd4d020 >2014-08-11 11:21:48,760 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-18) [494a81b4] FINISH, UpdateVmDynamicDataVDSCommand, log id: 7fd4d020 >2014-08-11 11:21:48,809 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-18) [494a81b4] Correlation ID: 494a81b4, Job ID: 9d3dc7cf-ad7b-42f9-a621-12db605c0cba, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm was initiated by User admin (Host: h1). >2014-08-11 11:21:48,810 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-18) [494a81b4] BaseAsyncTask::startPollingTask: Starting to poll task 97d9cc2a-1b59-40fe-a519-529410a665ff. >2014-08-11 11:21:48,811 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-18) [494a81b4] BaseAsyncTask::startPollingTask: Starting to poll task 160c52c5-c817-4098-839d-557ba6a23647. >2014-08-11 11:21:48,811 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-18) [494a81b4] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 11:21:50,256 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-80) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 11:21:50,273 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-80) SPMAsyncTask::PollTask: Polling task 97d9cc2a-1b59-40fe-a519-529410a665ff (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 11:21:50,300 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-80) BaseAsyncTask::onTaskEndSuccess: Task 97d9cc2a-1b59-40fe-a519-529410a665ff (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 11:21:50,301 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-80) Task with DB Task ID 04c75d26-6e1a-4919-9d60-a5a5eed8e46b and VDSM Task ID 160c52c5-c817-4098-839d-557ba6a23647 is in state Polling. End action for command f36529fb-4f41-43f2-bfaf-034634ca44b4 will proceed when all the entitys tasks are completed. >2014-08-11 11:21:50,302 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-80) SPMAsyncTask::PollTask: Polling task 160c52c5-c817-4098-839d-557ba6a23647 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-11 11:21:50,302 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-80) Finished polling Tasks, will poll again in 10 seconds. >2014-08-11 11:21:52,918 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) VM vm 14c0fa91-1814-438b-8d7a-4e7de1e1da25 moved from PreparingForHibernate --> Up >2014-08-11 11:22:00,322 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-100) BaseAsyncTask::onTaskEndSuccess: Task 97d9cc2a-1b59-40fe-a519-529410a665ff (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 11:22:00,323 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-100) Task with DB Task ID 04c75d26-6e1a-4919-9d60-a5a5eed8e46b and VDSM Task ID 160c52c5-c817-4098-839d-557ba6a23647 is in state Polling. End action for command f36529fb-4f41-43f2-bfaf-034634ca44b4 will proceed when all the entitys tasks are completed. >2014-08-11 11:22:00,323 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-100) SPMAsyncTask::PollTask: Polling task 160c52c5-c817-4098-839d-557ba6a23647 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 11:22:00,339 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-100) BaseAsyncTask::onTaskEndSuccess: Task 160c52c5-c817-4098-839d-557ba6a23647 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 11:22:00,340 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-100) CommandAsyncTask::endActionIfNecessary: All tasks of command f36529fb-4f41-43f2-bfaf-034634ca44b4 has ended -> executing endAction >2014-08-11 11:22:00,340 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-100) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: f36529fb-4f41-43f2-bfaf-034634ca44b4): calling endAction . >2014-08-11 11:22:00,341 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-35) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-11 11:22:00,372 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-35) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-11 11:22:00,374 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-35) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,181d3718-6f5a-43d4-beac-82a4631d87c5,aed9b3d4-8c5f-45d7-a3ee-864769c6eb42,b733c16d-9c36-42ce-93ea-c31d53fa2aaa,2db253d5-b04a-4991-8257-fd81190310ad), log id: 6c44e7f9 >2014-08-11 11:22:00,376 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-35) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,181d3718-6f5a-43d4-beac-82a4631d87c5,aed9b3d4-8c5f-45d7-a3ee-864769c6eb42,b733c16d-9c36-42ce-93ea-c31d53fa2aaa,2db253d5-b04a-4991-8257-fd81190310ad), log id: 1287448 >2014-08-11 11:22:00,384 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-35) FINISH, HibernateBrokerVDSCommand, log id: 1287448 >2014-08-11 11:22:00,397 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-35) FINISH, HibernateVDSCommand, log id: 6c44e7f9 >2014-08-11 11:22:00,398 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-35) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-11 11:22:00,399 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-35) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-11 11:22:00,405 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-35) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 160c52c5-c817-4098-839d-557ba6a23647 >2014-08-11 11:22:00,406 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 160c52c5-c817-4098-839d-557ba6a23647), log id: 2a72806f >2014-08-11 11:22:00,408 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=160c52c5-c817-4098-839d-557ba6a23647), log id: 434e0d2e >2014-08-11 11:22:00,597 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) FINISH, HSMClearTaskVDSCommand, log id: 434e0d2e >2014-08-11 11:22:00,597 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) FINISH, SPMClearTaskVDSCommand, log id: 2a72806f >2014-08-11 11:22:00,604 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-35) BaseAsyncTask::removeTaskFromDB: Removed task 160c52c5-c817-4098-839d-557ba6a23647 from DataBase >2014-08-11 11:22:00,604 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-35) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 97d9cc2a-1b59-40fe-a519-529410a665ff >2014-08-11 11:22:00,605 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 97d9cc2a-1b59-40fe-a519-529410a665ff), log id: 59400488 >2014-08-11 11:22:00,606 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=97d9cc2a-1b59-40fe-a519-529410a665ff), log id: 79e9aa7e >2014-08-11 11:22:00,730 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) FINISH, HSMClearTaskVDSCommand, log id: 79e9aa7e >2014-08-11 11:22:00,730 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) FINISH, SPMClearTaskVDSCommand, log id: 59400488 >2014-08-11 11:22:00,778 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-35) BaseAsyncTask::removeTaskFromDB: Removed task 97d9cc2a-1b59-40fe-a519-529410a665ff from DataBase >2014-08-11 11:22:00,778 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-35) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity f36529fb-4f41-43f2-bfaf-034634ca44b4 >2014-08-11 11:22:02,089 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-17) START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, force=false, secondsToWait=0, gracefully=false, reason=), log id: 39d8f6ea >2014-08-11 11:22:02,099 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, DestroyVDSCommand, log id: 39d8f6ea >2014-08-11 11:22:02,119 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm on Host h1 is suspended. >2014-08-11 11:22:02,121 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) VM vm (14c0fa91-1814-438b-8d7a-4e7de1e1da25) is running in db and not running in VDS h1 >2014-08-11 11:22:02,121 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-17) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 1110ada2 >2014-08-11 11:22:02,166 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-17) FINISH, FullListVdsCommand, return: [], log id: 1110ada2 >2014-08-11 11:22:02,201 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-44) [8ab2140] Running command: ProcessDownVmCommand internal: true. >2014-08-11 11:23:10,114 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-94) Setting new tasks map. The map contains now 0 tasks >2014-08-11 11:23:10,115 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-94) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 11:42:05,166 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-11 11:42:05,167 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-11 11:42:05,169 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-11 11:42:05,179 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-11 11:42:05,181 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-11 11:42:05,182 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-11 11:42:05,183 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-11 11:42:05,183 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-11 11:42:05,184 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-11 11:42:05,185 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-11 11:42:05,186 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-11 11:42:05,186 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-11 11:42:05,188 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-11 11:42:05,188 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-11 11:42:05,189 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-11 11:42:05,190 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-11 11:42:05,190 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-11 11:42:05,191 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-11 11:42:05,191 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-11 11:42:05,192 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-11 11:42:05,194 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-11 11:42:05,195 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-11 11:42:05,195 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_USER" is "engine". >2014-08-11 11:42:05,197 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-11 11:42:05,200 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-11 11:42:05,201 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-11 11:42:05,201 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-11 11:42:05,202 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-11 11:42:05,205 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-11 11:42:05,206 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-11 11:42:05,206 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-11 11:42:05,207 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-11 11:42:05,209 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-11 11:42:05,210 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-11 11:42:05,211 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-11 11:42:05,212 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-11 11:42:05,213 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-11 11:42:05,213 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-11 11:42:05,239 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-11 11:42:05,240 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-11 11:42:05,240 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-11 11:42:05,241 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-11 11:42:05,243 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-11 11:42:05,244 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-11 11:42:05,244 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-11 11:42:05,245 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-11 11:42:05,245 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-11 11:42:05,245 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-11 11:42:05,246 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-11 11:42:05,246 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-11 11:42:05,247 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-11 11:42:05,247 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-11 11:42:05,247 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-11 11:42:05,248 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-11 11:42:05,248 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-11 11:42:05,249 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-11 11:42:05,249 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-11 11:42:05,250 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-11 11:42:05,250 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-11 11:42:05,251 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-11 11:42:05,251 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-11 11:42:05,252 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-11 11:42:05,252 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-11 11:42:05,253 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_USER" is "ovirt". >2014-08-11 11:42:05,253 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-11 11:42:05,254 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-11 11:42:05,254 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-11 11:42:05,255 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-11 11:42:05,255 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-11 11:42:05,256 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-11 11:42:05,626 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-4) Start initializing Backend >2014-08-11 11:42:05,773 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-4) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 11:42:05,872 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-4) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 11:42:05,925 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-4) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-11 11:42:05,947 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-4) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-11 11:42:06,071 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-4) Running ovirt-engine 3.5.0.0 >2014-08-11 11:42:06,071 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-4) Start initializing dictionaries >2014-08-11 11:42:06,074 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-4) Finished initializing dictionaries >2014-08-11 11:42:06,075 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-4) Start initializing AuditLogCleanupManager >2014-08-11 11:42:06,075 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-4) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-11 11:42:06,091 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-4) Finished initializing AuditLogCleanupManager >2014-08-11 11:42:06,091 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-4) Start initializing CommandEntityCleanupManager >2014-08-11 11:42:06,092 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-4) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-11 11:42:06,093 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-4) Finished initializing CommandEntityCleanupManager >2014-08-11 11:42:06,095 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-4) Start initializing TagsDirector >2014-08-11 11:42:06,105 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-4) Tag root added to tree >2014-08-11 11:42:06,114 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-4) Finished initializing TagsDirector >2014-08-11 11:42:06,115 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-4) Start initializing IsoDomainListSyncronizer >2014-08-11 11:42:06,122 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-4) Finished initializing IsoDomainListSyncronizer >2014-08-11 11:42:06,198 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-4) Completed initializing handlers >2014-08-11 11:42:06,220 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-4) Start initializing ErrorTranslatorImpl >2014-08-11 11:42:06,232 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-4) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-11 11:42:06,234 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-4) Finished initializing ErrorTranslatorImpl >2014-08-11 11:42:06,234 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-4) Start initializing ErrorTranslatorImpl >2014-08-11 11:42:06,235 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-4) Finished initializing ErrorTranslatorImpl >2014-08-11 11:42:06,236 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-4) Mark incomplete jobs as UNKNOWN >2014-08-11 11:42:06,263 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-4) Start initializing JobRepositoryCleanupManager >2014-08-11 11:42:06,264 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-4) Finished initializing JobRepositoryCleanupManager >2014-08-11 11:42:06,264 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-4) Start initializing AutoRecoveryManager >2014-08-11 11:42:06,265 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-4) Finished initializing AutoRecoveryManager >2014-08-11 11:42:06,268 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-4) Start initializing ExecutionMessageDirector >2014-08-11 11:42:06,271 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-4) Finished initializing ExecutionMessageDirector >2014-08-11 11:42:06,327 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-4) Start of enabled extensions list >2014-08-11 11:42:06,328 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-4) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-11 11:42:06,329 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-4) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-11 11:42:06,330 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-4) End of enabled extensions list >2014-08-11 11:42:06,401 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-4) Start initializing DbUserCacheManager >2014-08-11 11:42:06,402 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-4) Finished initializing DbUserCacheManager >2014-08-11 11:42:06,419 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-4) Initialization of AsyncTaskManager completed successfully. >2014-08-11 11:42:06,422 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-4) Start initializing ResourceManager >2014-08-11 11:42:06,449 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-11 11:42:06,504 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-4) Entered VdsManager constructor >2014-08-11 11:42:06,528 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-4) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-11 11:42:06,637 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-4) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-11 11:42:06,638 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-4) Entered VdsManager constructor >2014-08-11 11:42:06,655 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-4) Initialize vdsBroker (galactica,54,321) >2014-08-11 11:42:06,695 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-4) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-11 11:42:06,716 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-4) Finished initializing ResourceManager >2014-08-11 11:42:06,721 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-4) Initialization of OvfDataUpdater completed successfully. >2014-08-11 11:42:06,722 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Start scheduling to enable vds load balancer >2014-08-11 11:42:06,723 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Finished scheduling to enable vds load balancer >2014-08-11 11:42:06,724 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Start HA Reservation check >2014-08-11 11:42:06,725 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Finished HA Reservation check >2014-08-11 11:42:06,744 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-4) Init VM custom properties utilities >2014-08-11 11:42:06,746 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-4) Init device custom properties utilities >2014-08-11 11:42:06,750 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Initializing Scheduling manager >2014-08-11 11:42:06,789 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) External scheduler disabled, discovery skipped >2014-08-11 11:42:06,790 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Initialized Scheduling manager >2014-08-11 11:42:06,791 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-4) Initializing DWH Heart Beat >2014-08-11 11:42:06,792 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-4) DWH Heart Beat initialized >2014-08-11 11:42:06,793 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-11 11:42:06,796 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-11 11:42:06,797 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-11 11:42:09,711 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: h2 >2014-08-11 11:42:10,307 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 57a912e0 >2014-08-11 11:42:10,314 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 57a912e0 >2014-08-11 11:42:10,320 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-11 11:42:10,324 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: UNASSIGNED not exist in string table >2014-08-11 11:42:10,325 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-11 11:42:10,325 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-11 11:42:10,326 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-11 11:42:10,327 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-11 11:42:10,327 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-11 11:42:10,328 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-11 11:42:10,329 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-11 11:42:10,329 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-11 11:42:10,330 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-11 11:42:10,331 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-11 11:42:10,331 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STOPPED not exist in string table >2014-08-11 11:42:10,332 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STARTED not exist in string table >2014-08-11 11:42:10,332 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_ERROR not exist in string table >2014-08-11 11:42:10,332 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-11 11:42:10,333 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-11 11:42:10,334 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-11 11:42:10,335 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-11 11:42:10,513 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-11 11:42:10,514 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is running with disabled SELinux. >2014-08-11 11:42:10,789 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h1 >2014-08-11 11:42:10,905 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [4c21321d] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-11 11:42:16,741 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-23) [624155e] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-11 11:42:16,775 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [624155e] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-11 11:42:16,782 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [624155e] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-11 11:42:16,849 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [624155e] Correlation ID: 624155e, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-11 11:42:16,961 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) [624155e] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-11 11:42:16,973 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) [624155e] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-11 11:42:16,998 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [624155e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-11 11:42:17,024 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [624155e] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 5d6ccc7b >2014-08-11 11:42:17,147 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [624155e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 52 GB of free space >2014-08-11 11:42:17,255 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [624155e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 52 GB of free space >2014-08-11 11:42:17,256 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [624155e] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-11 11:42:17,264 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [624155e] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 6265ce21 >2014-08-11 11:42:17,270 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [624155e] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 6265ce21 >2014-08-11 11:42:17,271 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [624155e] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 5d6ccc7b >2014-08-11 11:42:17,271 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [624155e] Discovered no tasks on Storage Pool Default >2014-08-11 11:42:46,500 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-2) Running command: LoginAdminUserCommand internal: false. >2014-08-11 11:42:46,526 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-11 11:42:50,479 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-2) Running command: LoginUserCommand internal: false. >2014-08-11 11:43:04,387 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [69d57ede] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 11:43:04,535 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [69d57ede] START, IsVmDuringInitiatingVDSCommand( vmId = 14c0fa91-1814-438b-8d7a-4e7de1e1da25), log id: 6f955741 >2014-08-11 11:43:04,536 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [69d57ede] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 6f955741 >2014-08-11 11:43:04,705 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-13) [69d57ede] Running command: RunVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 11:43:04,790 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-13) [69d57ede] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@115a1215), log id: 63aa7972 >2014-08-11 11:43:04,805 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-13) [69d57ede] FINISH, UpdateVmDynamicDataVDSCommand, log id: 63aa7972 >2014-08-11 11:43:04,838 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-13) [69d57ede] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, vm=VM [vm]), log id: 24330801 >2014-08-11 11:43:04,860 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-13) [69d57ede] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, vm=VM [vm]), log id: 358bd8b5 >2014-08-11 11:43:04,931 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-13) [69d57ede] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,hiberVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,181d3718-6f5a-43d4-beac-82a4631d87c5,aed9b3d4-8c5f-45d7-a3ee-864769c6eb42,b733c16d-9c36-42ce-93ea-c31d53fa2aaa,2db253d5-b04a-4991-8257-fd81190310ad,transparentHugePages=true,vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=68e21ff2-4b5a-4b29-8765-eec79f1b15dd}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=496deb19-46c7-4b58-94a3-06b7fa34f7ac}, {shared=false, index=0, volumeID=3f8b5bbe-741b-4a4f-9397-84123d530772, propagateErrors=off, format=raw, type=disk, iface=virtio, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, imageID=99e5cda5-262b-4469-b336-04e3f0cf7c38, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=99e5cda5-262b-4469-b336-04e3f0cf7c38}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:57, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=1e5b18b9-16eb-4b96-83b6-4568b8bed355}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=3959560b-27d1-4acf-acd9-c39f0c4148b2}],vmName=vm,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 11:43:05,121 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-13) [69d57ede] FINISH, CreateVDSCommand, log id: 358bd8b5 >2014-08-11 11:43:05,252 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-13) [69d57ede] FINISH, CreateVmVDSCommand, return: RestoringState, log id: 24330801 >2014-08-11 11:43:05,252 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-13) [69d57ede] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 11:43:05,326 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-13) [69d57ede] Correlation ID: 69d57ede, Job ID: 65758acc-47c1-49fe-b803-76d2c11622c6, Call Stack: null, Custom Event ID: -1, Message: VM vm was started by admin (Host: h1). >2014-08-11 11:43:06,165 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) VM vm 14c0fa91-1814-438b-8d7a-4e7de1e1da25 moved from RestoringState --> Up >2014-08-11 11:43:06,167 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-49) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 717cd79e >2014-08-11 11:43:06,179 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-49) FINISH, FullListVdsCommand, return: [{displaySecurePort=5902, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=25140, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@3b300047, vmName=vm, fileTransferEnable=true, cpuType=SandyBridge}], log id: 717cd79e >2014-08-11 11:43:06,192 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) Received a memballoon Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 11:43:06,194 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) Received a spice Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5902, type=graphics} >2014-08-11 11:43:06,420 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) Correlation ID: 69d57ede, Job ID: 65758acc-47c1-49fe-b803-76d2c11622c6, Call Stack: null, Custom Event ID: -1, Message: VM vm started on Host h1 >2014-08-11 11:43:06,439 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (DefaultQuartzScheduler_Worker-49) [203f5400] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-11 11:43:06,553 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-49) [203f5400] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 181d3718-6f5a-43d4-beac-82a4631d87c5, postZeros = false, forceDelete = false), log id: 56135323 >2014-08-11 11:43:07,251 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-49) [203f5400] FINISH, DeleteImageGroupVDSCommand, log id: 56135323 >2014-08-11 11:43:07,389 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-49) [203f5400] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 1ccbf3ab-b438-4beb-b063-02b842118038 >2014-08-11 11:43:07,390 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-49) [203f5400] CommandMultiAsyncTasks::AttachTask: Attaching task 1fa811be-b87f-45e9-b199-fec5ea17e5df to command 1ccbf3ab-b438-4beb-b063-02b842118038. >2014-08-11 11:43:07,413 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-49) [203f5400] Adding task 1fa811be-b87f-45e9-b199-fec5ea17e5df (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 11:43:07,455 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-49) [203f5400] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b733c16d-9c36-42ce-93ea-c31d53fa2aaa, postZeros = false, forceDelete = false), log id: 54c5339c >2014-08-11 11:43:08,029 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-49) [203f5400] FINISH, DeleteImageGroupVDSCommand, log id: 54c5339c >2014-08-11 11:43:08,093 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-49) [203f5400] CommandMultiAsyncTasks::AttachTask: Attaching task 4d0aab1f-595c-4221-9061-5c7ca2fe8558 to command 1ccbf3ab-b438-4beb-b063-02b842118038. >2014-08-11 11:43:08,216 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-49) [203f5400] Adding task 4d0aab1f-595c-4221-9061-5c7ca2fe8558 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 11:43:08,303 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) [203f5400] BaseAsyncTask::startPollingTask: Starting to poll task 1fa811be-b87f-45e9-b199-fec5ea17e5df. >2014-08-11 11:43:08,304 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) [203f5400] BaseAsyncTask::startPollingTask: Starting to poll task 4d0aab1f-595c-4221-9061-5c7ca2fe8558. >2014-08-11 11:43:08,307 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-49) [203f5400] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@704f6829), log id: 67260926 >2014-08-11 11:43:08,317 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-49) [203f5400] FINISH, UpdateVmDynamicDataVDSCommand, log id: 67260926 >2014-08-11 11:43:10,212 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [10cdfd7c] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 11:43:10,216 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [10cdfd7c] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:258) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:739) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view6.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 11:43:10,255 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [10cdfd7c] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 11:43:14,375 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-78) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 73c5b8d6 >2014-08-11 11:43:14,390 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-78) FINISH, FullListVdsCommand, return: [{displaySecurePort=5902, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=25140, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@3b21f871, vmName=vm, fileTransferEnable=true, cpuType=SandyBridge}], log id: 73c5b8d6 >2014-08-11 11:43:14,396 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Received a memballoon Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 11:43:14,397 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Received a spice Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5902, type=graphics} >2014-08-11 11:43:16,406 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-82) task id 4d0aab1f-595c-4221-9061-5c7ca2fe8558 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:43:16,407 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-82) task id 1fa811be-b87f-45e9-b199-fec5ea17e5df is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:43:19,628 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [8e74d6c] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 11:43:19,628 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [8e74d6c] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:258) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:739) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view6.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 11:43:19,657 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [8e74d6c] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 11:43:26,406 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-9) task id 4d0aab1f-595c-4221-9061-5c7ca2fe8558 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:43:26,406 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-9) task id 1fa811be-b87f-45e9-b199-fec5ea17e5df is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:43:36,405 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-37) task id 4d0aab1f-595c-4221-9061-5c7ca2fe8558 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:43:36,406 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-37) task id 1fa811be-b87f-45e9-b199-fec5ea17e5df is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:43:46,406 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-66) task id 4d0aab1f-595c-4221-9061-5c7ca2fe8558 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:43:46,407 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-66) task id 1fa811be-b87f-45e9-b199-fec5ea17e5df is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:43:56,412 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-91) task id 4d0aab1f-595c-4221-9061-5c7ca2fe8558 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:43:56,413 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-91) task id 1fa811be-b87f-45e9-b199-fec5ea17e5df is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:44:06,414 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-19) task id 4d0aab1f-595c-4221-9061-5c7ca2fe8558 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:44:06,414 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-19) task id 1fa811be-b87f-45e9-b199-fec5ea17e5df is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:44:16,415 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) task id 4d0aab1f-595c-4221-9061-5c7ca2fe8558 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:44:16,416 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) task id 4d0aab1f-595c-4221-9061-5c7ca2fe8558 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:44:16,417 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) task id 1fa811be-b87f-45e9-b199-fec5ea17e5df has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:44:16,418 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-51) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 11:44:16,418 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) task id 4d0aab1f-595c-4221-9061-5c7ca2fe8558 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:44:16,419 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) task id 1fa811be-b87f-45e9-b199-fec5ea17e5df has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:44:16,427 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) task id 4d0aab1f-595c-4221-9061-5c7ca2fe8558 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:44:16,427 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) SPMAsyncTask::PollTask: Polling task 4d0aab1f-595c-4221-9061-5c7ca2fe8558 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 11:44:16,442 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) BaseAsyncTask::onTaskEndSuccess: Task 4d0aab1f-595c-4221-9061-5c7ca2fe8558 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 11:44:16,443 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-51) Task with DB Task ID b3c2dc73-f3fc-4d31-892a-8193ff8f5119 and VDSM Task ID 1fa811be-b87f-45e9-b199-fec5ea17e5df is in state Polling. End action for command 1ccbf3ab-b438-4beb-b063-02b842118038 will proceed when all the entitys tasks are completed. >2014-08-11 11:44:16,444 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) task id 1fa811be-b87f-45e9-b199-fec5ea17e5df has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 11:44:16,445 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) SPMAsyncTask::PollTask: Polling task 1fa811be-b87f-45e9-b199-fec5ea17e5df (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 11:44:16,459 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) BaseAsyncTask::onTaskEndSuccess: Task 1fa811be-b87f-45e9-b199-fec5ea17e5df (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 11:44:16,460 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-51) CommandAsyncTask::endActionIfNecessary: All tasks of command 1ccbf3ab-b438-4beb-b063-02b842118038 has ended -> executing endAction >2014-08-11 11:44:16,461 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-51) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 1ccbf3ab-b438-4beb-b063-02b842118038): calling endAction . >2014-08-11 11:44:16,462 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-4) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RunVm, executionIndex: 0 >2014-08-11 11:44:16,484 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-4) Ending command successfully: org.ovirt.engine.core.bll.RunVmCommand >2014-08-11 11:44:16,508 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-4) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm completed, handling the result. >2014-08-11 11:44:16,509 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-4) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm succeeded, clearing tasks. >2014-08-11 11:44:16,517 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-4) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 4d0aab1f-595c-4221-9061-5c7ca2fe8558 >2014-08-11 11:44:16,519 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-4) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 4d0aab1f-595c-4221-9061-5c7ca2fe8558), log id: 5122b7f6 >2014-08-11 11:44:16,523 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-4) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=4d0aab1f-595c-4221-9061-5c7ca2fe8558), log id: 43b5a5db >2014-08-11 11:44:16,595 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-4) FINISH, HSMClearTaskVDSCommand, log id: 43b5a5db >2014-08-11 11:44:16,595 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-4) FINISH, SPMClearTaskVDSCommand, log id: 5122b7f6 >2014-08-11 11:44:16,616 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-4) BaseAsyncTask::removeTaskFromDB: Removed task 4d0aab1f-595c-4221-9061-5c7ca2fe8558 from DataBase >2014-08-11 11:44:16,617 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-4) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 1fa811be-b87f-45e9-b199-fec5ea17e5df >2014-08-11 11:44:16,618 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-4) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 1fa811be-b87f-45e9-b199-fec5ea17e5df), log id: 25f7c122 >2014-08-11 11:44:16,619 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-4) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=1fa811be-b87f-45e9-b199-fec5ea17e5df), log id: 2ca8b2a7 >2014-08-11 11:44:16,669 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-4) FINISH, HSMClearTaskVDSCommand, log id: 2ca8b2a7 >2014-08-11 11:44:16,670 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-4) FINISH, SPMClearTaskVDSCommand, log id: 25f7c122 >2014-08-11 11:44:16,682 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-4) BaseAsyncTask::removeTaskFromDB: Removed task 1fa811be-b87f-45e9-b199-fec5ea17e5df from DataBase >2014-08-11 11:44:16,683 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-4) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 1ccbf3ab-b438-4beb-b063-02b842118038 >2014-08-11 11:45:36,407 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-69) Setting new tasks map. The map contains now 0 tasks >2014-08-11 11:45:36,408 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-69) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 11:56:37,488 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [4b9391b6] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 11:56:58,573 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] Running command: HibernateVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 11:56:58,608 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@c544bff2), log id: 51bbb318 >2014-08-11 11:56:58,610 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] FINISH, UpdateVmDynamicDataVDSCommand, log id: 51bbb318 >2014-08-11 11:56:58,623 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = d058e31b-5ab3-43e9-8dfa-9a5ae870c72d, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = 881f3dd8-505a-45b5-b870-feebb0adcf82, newImageDescription = ), log id: 535a1d39 >2014-08-11 11:56:58,629 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 11:56:59,056 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] FINISH, CreateImageVDSCommand, return: 881f3dd8-505a-45b5-b870-feebb0adcf82, log id: 535a1d39 >2014-08-11 11:56:59,183 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 7d26d0c8-bf2d-4a29-ab6d-602bb4787175 >2014-08-11 11:56:59,184 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] CommandMultiAsyncTasks::AttachTask: Attaching task 8368996c-a24b-40bd-9713-6145674a5ce5 to command 7d26d0c8-bf2d-4a29-ab6d-602bb4787175. >2014-08-11 11:56:59,202 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] Adding task 8368996c-a24b-40bd-9713-6145674a5ce5 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 11:56:59,221 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = e42ad480-ec06-4187-804e-5fff277699c7, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = 38108903-bc69-4faa-81f3-76957ad5fd4d, newImageDescription = ), log id: 51f28e02 >2014-08-11 11:56:59,222 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 11:56:59,738 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] FINISH, CreateImageVDSCommand, return: 38108903-bc69-4faa-81f3-76957ad5fd4d, log id: 51f28e02 >2014-08-11 11:56:59,800 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] CommandMultiAsyncTasks::AttachTask: Attaching task f8ef93d6-0341-404d-b4b3-6e544c9685c5 to command 7d26d0c8-bf2d-4a29-ab6d-602bb4787175. >2014-08-11 11:56:59,816 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] Adding task f8ef93d6-0341-404d-b4b3-6e544c9685c5 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 11:56:59,826 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@ae069401), log id: 71b230b3 >2014-08-11 11:56:59,833 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] FINISH, UpdateVmDynamicDataVDSCommand, log id: 71b230b3 >2014-08-11 11:56:59,866 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] Correlation ID: 4b9391b6, Job ID: dc9abd21-8167-4a08-889e-27c867d7152a, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm was initiated by User admin (Host: h1). >2014-08-11 11:56:59,867 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] BaseAsyncTask::startPollingTask: Starting to poll task 8368996c-a24b-40bd-9713-6145674a5ce5. >2014-08-11 11:56:59,867 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] BaseAsyncTask::startPollingTask: Starting to poll task f8ef93d6-0341-404d-b4b3-6e544c9685c5. >2014-08-11 11:56:59,868 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-32) [4b9391b6] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 11:57:00,834 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-42) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 11:57:00,844 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-42) SPMAsyncTask::PollTask: Polling task f8ef93d6-0341-404d-b4b3-6e544c9685c5 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-11 11:57:00,845 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-42) SPMAsyncTask::PollTask: Polling task 8368996c-a24b-40bd-9713-6145674a5ce5 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-11 11:57:00,846 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-42) Finished polling Tasks, will poll again in 10 seconds. >2014-08-11 11:57:01,466 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) VM vm 14c0fa91-1814-438b-8d7a-4e7de1e1da25 moved from PreparingForHibernate --> Up >2014-08-11 11:57:10,852 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) SPMAsyncTask::PollTask: Polling task f8ef93d6-0341-404d-b4b3-6e544c9685c5 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 11:57:10,873 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) BaseAsyncTask::onTaskEndSuccess: Task f8ef93d6-0341-404d-b4b3-6e544c9685c5 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 11:57:10,874 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-79) Task with DB Task ID 1ceb3cb9-ac82-4eb9-bc75-3ccd218d2120 and VDSM Task ID 8368996c-a24b-40bd-9713-6145674a5ce5 is in state Polling. End action for command 7d26d0c8-bf2d-4a29-ab6d-602bb4787175 will proceed when all the entitys tasks are completed. >2014-08-11 11:57:10,875 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) SPMAsyncTask::PollTask: Polling task 8368996c-a24b-40bd-9713-6145674a5ce5 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 11:57:10,890 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) BaseAsyncTask::onTaskEndSuccess: Task 8368996c-a24b-40bd-9713-6145674a5ce5 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 11:57:10,890 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-79) CommandAsyncTask::endActionIfNecessary: All tasks of command 7d26d0c8-bf2d-4a29-ab6d-602bb4787175 has ended -> executing endAction >2014-08-11 11:57:10,891 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-79) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 7d26d0c8-bf2d-4a29-ab6d-602bb4787175): calling endAction . >2014-08-11 11:57:10,892 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-3) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-11 11:57:10,939 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-3) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-11 11:57:10,974 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-3) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,d058e31b-5ab3-43e9-8dfa-9a5ae870c72d,881f3dd8-505a-45b5-b870-feebb0adcf82,e42ad480-ec06-4187-804e-5fff277699c7,38108903-bc69-4faa-81f3-76957ad5fd4d), log id: 59610fb4 >2014-08-11 11:57:10,976 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-3) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,d058e31b-5ab3-43e9-8dfa-9a5ae870c72d,881f3dd8-505a-45b5-b870-feebb0adcf82,e42ad480-ec06-4187-804e-5fff277699c7,38108903-bc69-4faa-81f3-76957ad5fd4d), log id: 754b4d1a >2014-08-11 11:57:10,982 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-3) FINISH, HibernateBrokerVDSCommand, log id: 754b4d1a >2014-08-11 11:57:11,001 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-3) FINISH, HibernateVDSCommand, log id: 59610fb4 >2014-08-11 11:57:11,002 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-3) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-11 11:57:11,003 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-3) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-11 11:57:11,014 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-3) SPMAsyncTask::ClearAsyncTask: Attempting to clear task f8ef93d6-0341-404d-b4b3-6e544c9685c5 >2014-08-11 11:57:11,015 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-3) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = f8ef93d6-0341-404d-b4b3-6e544c9685c5), log id: 1ed4785b >2014-08-11 11:57:11,017 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-3) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=f8ef93d6-0341-404d-b4b3-6e544c9685c5), log id: 796a1421 >2014-08-11 11:57:11,115 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-3) FINISH, HSMClearTaskVDSCommand, log id: 796a1421 >2014-08-11 11:57:11,116 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-3) FINISH, SPMClearTaskVDSCommand, log id: 1ed4785b >2014-08-11 11:57:11,130 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-3) BaseAsyncTask::removeTaskFromDB: Removed task f8ef93d6-0341-404d-b4b3-6e544c9685c5 from DataBase >2014-08-11 11:57:11,130 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-3) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 8368996c-a24b-40bd-9713-6145674a5ce5 >2014-08-11 11:57:11,131 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-3) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 8368996c-a24b-40bd-9713-6145674a5ce5), log id: 787bdb03 >2014-08-11 11:57:11,132 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-3) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=8368996c-a24b-40bd-9713-6145674a5ce5), log id: b67dfc1 >2014-08-11 11:57:11,415 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-3) FINISH, HSMClearTaskVDSCommand, log id: b67dfc1 >2014-08-11 11:57:11,415 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-3) FINISH, SPMClearTaskVDSCommand, log id: 787bdb03 >2014-08-11 11:57:11,428 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-3) BaseAsyncTask::removeTaskFromDB: Removed task 8368996c-a24b-40bd-9713-6145674a5ce5 from DataBase >2014-08-11 11:57:11,428 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-3) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 7d26d0c8-bf2d-4a29-ab6d-602bb4787175 >2014-08-11 11:57:13,742 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-88) START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, force=false, secondsToWait=0, gracefully=false, reason=), log id: 5b8901a1 >2014-08-11 11:57:13,753 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-88) FINISH, DestroyVDSCommand, log id: 5b8901a1 >2014-08-11 11:57:13,821 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-88) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm on Host h1 is suspended. >2014-08-11 11:57:13,822 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) VM vm (14c0fa91-1814-438b-8d7a-4e7de1e1da25) is running in db and not running in VDS h1 >2014-08-11 11:57:13,823 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-88) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 738d3d47 >2014-08-11 11:57:13,827 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-88) FINISH, FullListVdsCommand, return: [], log id: 738d3d47 >2014-08-11 11:57:13,921 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-12) [1d333657] Running command: ProcessDownVmCommand internal: true. >2014-08-11 13:35:04,271 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-11 13:35:04,272 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-11 13:35:04,273 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-11 13:35:04,274 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-11 13:35:04,275 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-11 13:35:04,276 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-11 13:35:04,277 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-11 13:35:04,278 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-11 13:35:04,278 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-11 13:35:04,279 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-11 13:35:04,280 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-11 13:35:04,280 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-11 13:35:04,281 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-11 13:35:04,281 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-11 13:35:04,282 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-11 13:35:04,282 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-11 13:35:04,283 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-11 13:35:04,283 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-11 13:35:04,284 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-11 13:35:04,284 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-11 13:35:04,285 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-11 13:35:04,286 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-11 13:35:04,286 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_USER" is "engine". >2014-08-11 13:35:04,287 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-11 13:35:04,287 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-11 13:35:04,288 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-11 13:35:04,288 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-11 13:35:04,289 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-11 13:35:04,290 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-11 13:35:04,291 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-11 13:35:04,292 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-11 13:35:04,292 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-11 13:35:04,293 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-11 13:35:04,293 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-11 13:35:04,294 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-11 13:35:04,294 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-11 13:35:04,296 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-11 13:35:04,296 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-11 13:35:04,298 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-11 13:35:04,298 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-11 13:35:04,299 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-11 13:35:04,299 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-11 13:35:04,300 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-11 13:35:04,300 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-11 13:35:04,301 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-11 13:35:04,302 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-11 13:35:04,302 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-11 13:35:04,303 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-11 13:35:04,304 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-11 13:35:04,304 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-11 13:35:04,305 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-11 13:35:04,305 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-11 13:35:04,306 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-11 13:35:04,307 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-11 13:35:04,308 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-11 13:35:04,308 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-11 13:35:04,309 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-11 13:35:04,309 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-11 13:35:04,310 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-11 13:35:04,310 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-11 13:35:04,311 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-11 13:35:04,312 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-11 13:35:04,312 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-11 13:35:04,313 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_USER" is "ovirt". >2014-08-11 13:35:04,313 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-11 13:35:04,314 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-11 13:35:04,315 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-11 13:35:04,315 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-11 13:35:04,316 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-11 13:35:04,317 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-11 13:35:04,686 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-6) Start initializing Backend >2014-08-11 13:35:04,805 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-6) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 13:35:04,897 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-6) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 13:35:04,934 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-6) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-11 13:35:04,946 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-6) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-11 13:35:05,076 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-6) Running ovirt-engine 3.5.0.0 >2014-08-11 13:35:05,076 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-6) Start initializing dictionaries >2014-08-11 13:35:05,080 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-6) Finished initializing dictionaries >2014-08-11 13:35:05,080 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Start initializing AuditLogCleanupManager >2014-08-11 13:35:05,081 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-11 13:35:05,098 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Finished initializing AuditLogCleanupManager >2014-08-11 13:35:05,098 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Start initializing CommandEntityCleanupManager >2014-08-11 13:35:05,099 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-11 13:35:05,100 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Finished initializing CommandEntityCleanupManager >2014-08-11 13:35:05,102 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-6) Start initializing TagsDirector >2014-08-11 13:35:05,112 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-6) Tag root added to tree >2014-08-11 13:35:05,120 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-6) Finished initializing TagsDirector >2014-08-11 13:35:05,121 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-6) Start initializing IsoDomainListSyncronizer >2014-08-11 13:35:05,126 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-6) Finished initializing IsoDomainListSyncronizer >2014-08-11 13:35:05,196 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-6) Completed initializing handlers >2014-08-11 13:35:05,215 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Start initializing ErrorTranslatorImpl >2014-08-11 13:35:05,227 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-11 13:35:05,228 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Finished initializing ErrorTranslatorImpl >2014-08-11 13:35:05,228 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Start initializing ErrorTranslatorImpl >2014-08-11 13:35:05,229 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Finished initializing ErrorTranslatorImpl >2014-08-11 13:35:05,229 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-6) Mark incomplete jobs as UNKNOWN >2014-08-11 13:35:05,271 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-6) Start initializing JobRepositoryCleanupManager >2014-08-11 13:35:05,272 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-6) Finished initializing JobRepositoryCleanupManager >2014-08-11 13:35:05,272 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-6) Start initializing AutoRecoveryManager >2014-08-11 13:35:05,274 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-6) Finished initializing AutoRecoveryManager >2014-08-11 13:35:05,276 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-6) Start initializing ExecutionMessageDirector >2014-08-11 13:35:05,277 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-6) Finished initializing ExecutionMessageDirector >2014-08-11 13:35:05,317 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-6) Start of enabled extensions list >2014-08-11 13:35:05,318 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-6) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-11 13:35:05,319 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-6) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-11 13:35:05,319 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-6) End of enabled extensions list >2014-08-11 13:35:05,649 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-6) Start initializing DbUserCacheManager >2014-08-11 13:35:05,650 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-6) Finished initializing DbUserCacheManager >2014-08-11 13:35:05,660 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-6) Initialization of AsyncTaskManager completed successfully. >2014-08-11 13:35:05,665 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-6) Start initializing ResourceManager >2014-08-11 13:35:05,681 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-11 13:35:05,748 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-6) Entered VdsManager constructor >2014-08-11 13:35:05,768 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-6) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-11 13:35:05,860 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-6) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-11 13:35:05,860 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-6) Entered VdsManager constructor >2014-08-11 13:35:05,870 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-6) Initialize vdsBroker (galactica,54,321) >2014-08-11 13:35:05,894 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-6) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-11 13:35:05,911 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-6) Finished initializing ResourceManager >2014-08-11 13:35:05,917 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-6) Initialization of OvfDataUpdater completed successfully. >2014-08-11 13:35:05,918 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Start scheduling to enable vds load balancer >2014-08-11 13:35:05,919 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Finished scheduling to enable vds load balancer >2014-08-11 13:35:05,919 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Start HA Reservation check >2014-08-11 13:35:05,920 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Finished HA Reservation check >2014-08-11 13:35:05,935 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-6) Init VM custom properties utilities >2014-08-11 13:35:05,935 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-6) Init device custom properties utilities >2014-08-11 13:35:05,940 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Initializing Scheduling manager >2014-08-11 13:35:05,959 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-11 13:35:05,960 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-11 13:35:05,961 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-11 13:35:05,971 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) External scheduler disabled, discovery skipped >2014-08-11 13:35:05,972 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Initialized Scheduling manager >2014-08-11 13:35:05,972 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-6) Initializing DWH Heart Beat >2014-08-11 13:35:05,974 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-6) DWH Heart Beat initialized >2014-08-11 13:35:08,910 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: h2 >2014-08-11 13:35:10,980 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 6cdd3098 >2014-08-11 13:35:10,997 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 6cdd3098 >2014-08-11 13:35:11,005 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-11 13:35:11,012 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: UNASSIGNED not exist in string table >2014-08-11 13:35:11,012 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-11 13:35:11,013 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-11 13:35:11,013 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-11 13:35:11,014 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-11 13:35:11,015 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-11 13:35:11,015 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-11 13:35:11,015 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-11 13:35:11,016 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-11 13:35:11,017 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-11 13:35:11,017 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-11 13:35:11,017 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STOPPED not exist in string table >2014-08-11 13:35:11,018 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STARTED not exist in string table >2014-08-11 13:35:11,018 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_ERROR not exist in string table >2014-08-11 13:35:11,018 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-11 13:35:11,019 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-11 13:35:11,019 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-11 13:35:11,020 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-11 13:35:11,208 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-11 13:35:11,209 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is running with disabled SELinux. >2014-08-11 13:35:11,513 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h1 >2014-08-11 13:35:11,624 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [63ac79f6] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-11 13:35:15,932 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-22) [ff941e9] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-11 13:35:15,938 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-22) [ff941e9] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-11 13:35:15,959 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-22) [ff941e9] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-11 13:35:16,012 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [ff941e9] Correlation ID: ff941e9, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-11 13:35:16,096 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-22) [ff941e9] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-11 13:35:16,112 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-22) [ff941e9] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-11 13:35:16,128 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [ff941e9] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-11 13:35:16,147 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [ff941e9] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 39ef2f53 >2014-08-11 13:35:16,203 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [ff941e9] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 52 GB of free space >2014-08-11 13:35:16,236 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [ff941e9] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 52 GB of free space >2014-08-11 13:35:16,237 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [ff941e9] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-11 13:35:16,243 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [ff941e9] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 38b5b6a1 >2014-08-11 13:35:16,251 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [ff941e9] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 38b5b6a1 >2014-08-11 13:35:16,252 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [ff941e9] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 39ef2f53 >2014-08-11 13:35:16,252 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [ff941e9] Discovered no tasks on Storage Pool Default >2014-08-11 13:35:18,817 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-4) Running command: LoginAdminUserCommand internal: false. >2014-08-11 13:35:18,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-11 13:35:22,499 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-3) Running command: LoginUserCommand internal: false. >2014-08-11 13:35:37,150 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-4) [1256c63c] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 13:35:37,368 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [1256c63c] START, IsVmDuringInitiatingVDSCommand( vmId = 14c0fa91-1814-438b-8d7a-4e7de1e1da25), log id: 7004e374 >2014-08-11 13:35:37,369 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [1256c63c] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 7004e374 >2014-08-11 13:35:37,596 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-39) [1256c63c] Running command: RunVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 13:35:37,672 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-39) [1256c63c] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@ceb20987), log id: 4fecd7a0 >2014-08-11 13:35:37,687 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-39) [1256c63c] FINISH, UpdateVmDynamicDataVDSCommand, log id: 4fecd7a0 >2014-08-11 13:35:37,719 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-39) [1256c63c] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, vm=VM [vm]), log id: 11781ce5 >2014-08-11 13:35:37,746 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-39) [1256c63c] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, vm=VM [vm]), log id: b7654da >2014-08-11 13:35:37,823 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-39) [1256c63c] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,hiberVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,d058e31b-5ab3-43e9-8dfa-9a5ae870c72d,881f3dd8-505a-45b5-b870-feebb0adcf82,e42ad480-ec06-4187-804e-5fff277699c7,38108903-bc69-4faa-81f3-76957ad5fd4d,transparentHugePages=true,vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=68e21ff2-4b5a-4b29-8765-eec79f1b15dd}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=496deb19-46c7-4b58-94a3-06b7fa34f7ac}, {shared=false, index=0, volumeID=3f8b5bbe-741b-4a4f-9397-84123d530772, propagateErrors=off, format=raw, type=disk, iface=virtio, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, imageID=99e5cda5-262b-4469-b336-04e3f0cf7c38, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=99e5cda5-262b-4469-b336-04e3f0cf7c38}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:57, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=1e5b18b9-16eb-4b96-83b6-4568b8bed355}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=3959560b-27d1-4acf-acd9-c39f0c4148b2}],vmName=vm,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 13:35:37,894 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-39) [1256c63c] FINISH, CreateVDSCommand, log id: b7654da >2014-08-11 13:35:37,910 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-39) [1256c63c] FINISH, CreateVmVDSCommand, return: RestoringState, log id: 11781ce5 >2014-08-11 13:35:37,911 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-39) [1256c63c] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 13:35:37,935 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-39) [1256c63c] Correlation ID: 1256c63c, Job ID: 9b11a5d4-8836-4fa8-889e-c5ea4c96c91e, Call Stack: null, Custom Event ID: -1, Message: VM vm was started by admin (Host: h1). >2014-08-11 13:35:39,657 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) VM vm 14c0fa91-1814-438b-8d7a-4e7de1e1da25 moved from RestoringState --> Up >2014-08-11 13:35:39,659 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-85) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 726aa87b >2014-08-11 13:35:39,682 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-85) FINISH, FullListVdsCommand, return: [{displaySecurePort=5902, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=1697, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@bfca19e, vmName=vm, fileTransferEnable=true, cpuType=SandyBridge}], log id: 726aa87b >2014-08-11 13:35:39,692 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Received a memballoon Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 13:35:39,693 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Received a spice Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5902, type=graphics} >2014-08-11 13:35:39,832 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-85) Correlation ID: 1256c63c, Job ID: 9b11a5d4-8836-4fa8-889e-c5ea4c96c91e, Call Stack: null, Custom Event ID: -1, Message: VM vm started on Host h1 >2014-08-11 13:35:39,852 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (DefaultQuartzScheduler_Worker-85) [335cba5b] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-11 13:35:39,966 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-85) [335cba5b] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = d058e31b-5ab3-43e9-8dfa-9a5ae870c72d, postZeros = false, forceDelete = false), log id: 24f50660 >2014-08-11 13:35:40,486 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-85) [335cba5b] FINISH, DeleteImageGroupVDSCommand, log id: 24f50660 >2014-08-11 13:35:40,555 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-85) [335cba5b] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 29b65bc4-d788-44a2-b4fe-243af3bec94a >2014-08-11 13:35:40,556 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-85) [335cba5b] CommandMultiAsyncTasks::AttachTask: Attaching task f63d6e98-9be6-4292-b0c1-136cf467cd06 to command 29b65bc4-d788-44a2-b4fe-243af3bec94a. >2014-08-11 13:35:40,586 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-85) [335cba5b] Adding task f63d6e98-9be6-4292-b0c1-136cf467cd06 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 13:35:40,610 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-85) [335cba5b] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = e42ad480-ec06-4187-804e-5fff277699c7, postZeros = false, forceDelete = false), log id: 9ade63a >2014-08-11 13:35:41,171 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-85) [335cba5b] FINISH, DeleteImageGroupVDSCommand, log id: 9ade63a >2014-08-11 13:35:41,192 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-85) [335cba5b] CommandMultiAsyncTasks::AttachTask: Attaching task e69b873a-8899-44d6-9463-26384d65fecc to command 29b65bc4-d788-44a2-b4fe-243af3bec94a. >2014-08-11 13:35:41,215 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-85) [335cba5b] Adding task e69b873a-8899-44d6-9463-26384d65fecc (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 13:35:41,258 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-85) [335cba5b] BaseAsyncTask::startPollingTask: Starting to poll task f63d6e98-9be6-4292-b0c1-136cf467cd06. >2014-08-11 13:35:41,259 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-85) [335cba5b] BaseAsyncTask::startPollingTask: Starting to poll task e69b873a-8899-44d6-9463-26384d65fecc. >2014-08-11 13:35:41,261 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-85) [335cba5b] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@840271d1), log id: 7862cea9 >2014-08-11 13:35:41,273 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-85) [335cba5b] FINISH, UpdateVmDynamicDataVDSCommand, log id: 7862cea9 >2014-08-11 13:35:44,385 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-93) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 2b6763fc >2014-08-11 13:35:44,398 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-93) FINISH, FullListVdsCommand, return: [{displaySecurePort=5902, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=1697, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@1d55b2e4, vmName=vm, fileTransferEnable=true, cpuType=SandyBridge}], log id: 2b6763fc >2014-08-11 13:35:44,407 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Received a memballoon Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 13:35:44,409 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Received a spice Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5902, type=graphics} >2014-08-11 13:35:45,655 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-97) task id f63d6e98-9be6-4292-b0c1-136cf467cd06 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:35:45,656 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-97) task id e69b873a-8899-44d6-9463-26384d65fecc is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:35:50,436 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [1359bebd] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 13:35:50,437 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [1359bebd] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:258) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:739) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view6.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 13:35:50,464 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [1359bebd] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 13:35:55,655 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-23) task id f63d6e98-9be6-4292-b0c1-136cf467cd06 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:35:55,657 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-23) task id e69b873a-8899-44d6-9463-26384d65fecc is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:05,659 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) task id f63d6e98-9be6-4292-b0c1-136cf467cd06 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:05,659 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-51) task id e69b873a-8899-44d6-9463-26384d65fecc is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:15,661 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-84) task id f63d6e98-9be6-4292-b0c1-136cf467cd06 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:15,662 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-84) task id e69b873a-8899-44d6-9463-26384d65fecc is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:25,664 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-11) task id f63d6e98-9be6-4292-b0c1-136cf467cd06 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:25,666 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-11) task id e69b873a-8899-44d6-9463-26384d65fecc is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:26,792 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [664f2e41] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 13:36:38,221 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) [ff941e9] task id f63d6e98-9be6-4292-b0c1-136cf467cd06 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:38,225 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) [ff941e9] task id e69b873a-8899-44d6-9463-26384d65fecc is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:56,415 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-36) task id f63d6e98-9be6-4292-b0c1-136cf467cd06 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:56,417 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-36) task id f63d6e98-9be6-4292-b0c1-136cf467cd06 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:56,418 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-36) task id e69b873a-8899-44d6-9463-26384d65fecc has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:56,420 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-36) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 13:36:56,421 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-36) task id f63d6e98-9be6-4292-b0c1-136cf467cd06 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:56,423 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-36) task id e69b873a-8899-44d6-9463-26384d65fecc has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:59,584 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-36) task id f63d6e98-9be6-4292-b0c1-136cf467cd06 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:59,585 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-36) SPMAsyncTask::PollTask: Polling task f63d6e98-9be6-4292-b0c1-136cf467cd06 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 13:36:59,600 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-36) BaseAsyncTask::onTaskEndSuccess: Task f63d6e98-9be6-4292-b0c1-136cf467cd06 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 13:36:59,601 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-36) Task with DB Task ID 510f43ff-89de-4de5-ba26-cb64b1deca65 and VDSM Task ID e69b873a-8899-44d6-9463-26384d65fecc is in state Polling. End action for command 29b65bc4-d788-44a2-b4fe-243af3bec94a will proceed when all the entitys tasks are completed. >2014-08-11 13:36:59,602 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-36) task id e69b873a-8899-44d6-9463-26384d65fecc has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:36:59,603 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-36) SPMAsyncTask::PollTask: Polling task e69b873a-8899-44d6-9463-26384d65fecc (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 13:36:59,617 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-36) BaseAsyncTask::onTaskEndSuccess: Task e69b873a-8899-44d6-9463-26384d65fecc (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 13:36:59,618 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-36) CommandAsyncTask::endActionIfNecessary: All tasks of command 29b65bc4-d788-44a2-b4fe-243af3bec94a has ended -> executing endAction >2014-08-11 13:36:59,618 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-36) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 29b65bc4-d788-44a2-b4fe-243af3bec94a): calling endAction . >2014-08-11 13:36:59,780 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RunVm, executionIndex: 0 >2014-08-11 13:37:04,802 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-50) Ending command successfully: org.ovirt.engine.core.bll.RunVmCommand >2014-08-11 13:37:04,804 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [664f2e41] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:258) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:739) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view6.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 13:37:04,873 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm completed, handling the result. >2014-08-11 13:37:04,935 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm succeeded, clearing tasks. >2014-08-11 13:37:04,949 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) SPMAsyncTask::ClearAsyncTask: Attempting to clear task f63d6e98-9be6-4292-b0c1-136cf467cd06 >2014-08-11 13:37:04,951 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = f63d6e98-9be6-4292-b0c1-136cf467cd06), log id: 22013def >2014-08-11 13:37:04,955 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=f63d6e98-9be6-4292-b0c1-136cf467cd06), log id: 411d6dd2 >2014-08-11 13:37:05,169 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) FINISH, HSMClearTaskVDSCommand, log id: 411d6dd2 >2014-08-11 13:37:05,170 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) FINISH, SPMClearTaskVDSCommand, log id: 22013def >2014-08-11 13:37:06,849 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [664f2e41] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 13:37:06,968 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) BaseAsyncTask::removeTaskFromDB: Removed task f63d6e98-9be6-4292-b0c1-136cf467cd06 from DataBase >2014-08-11 13:37:06,971 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) SPMAsyncTask::ClearAsyncTask: Attempting to clear task e69b873a-8899-44d6-9463-26384d65fecc >2014-08-11 13:37:07,009 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = e69b873a-8899-44d6-9463-26384d65fecc), log id: 75fa5f6c >2014-08-11 13:37:07,255 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=e69b873a-8899-44d6-9463-26384d65fecc), log id: 88ee7f7 >2014-08-11 13:37:09,729 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) FINISH, HSMClearTaskVDSCommand, log id: 88ee7f7 >2014-08-11 13:37:09,730 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) FINISH, SPMClearTaskVDSCommand, log id: 75fa5f6c >2014-08-11 13:37:09,795 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) BaseAsyncTask::removeTaskFromDB: Removed task e69b873a-8899-44d6-9463-26384d65fecc from DataBase >2014-08-11 13:37:09,796 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 29b65bc4-d788-44a2-b4fe-243af3bec94a >2014-08-11 13:38:01,589 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (http--0.0.0.0-8080-4) [1e891ef3] Lock Acquired to object EngineLock [exclusiveLocks= key: vm111 value: VM_NAME >, sharedLocks= ] >2014-08-11 13:38:01,628 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (http--0.0.0.0-8080-4) [1e891ef3] Running command: AddVmFromScratchCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-00000000039f Type: VdsGroupsAction group CREATE_VM with role type USER >2014-08-11 13:38:01,804 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [1e891ef3] Correlation ID: 1e891ef3, Job ID: cc5be160-1b70-425d-87b4-0a0a20fc46ba, Call Stack: null, Custom Event ID: -1, Message: VM vm111 was created by admin. >2014-08-11 13:38:01,821 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (http--0.0.0.0-8080-4) [1e891ef3] Lock freed to object EngineLock [exclusiveLocks= key: vm111 value: VM_NAME >, sharedLocks= ] >2014-08-11 13:38:02,360 INFO [org.ovirt.engine.core.bll.network.vm.AddVmInterfaceCommand] (http--0.0.0.0-8080-4) [6e09a6b1] Running command: AddVmInterfaceCommand internal: false. Entities affected : ID: 2e77d03b-978a-41d9-a9e9-a43c7035366b Type: VMAction group CONFIGURE_VM_NETWORK with role type USER, ID: 0000000e-000e-000e-000e-000000000181 Type: VnicProfileAction group CONFIGURE_VM_NETWORK with role type USER >2014-08-11 13:38:02,453 INFO [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (http--0.0.0.0-8080-4) [190609a0] Running command: ActivateDeactivateVmNicCommand internal: true. Entities affected : ID: 2e77d03b-978a-41d9-a9e9-a43c7035366b Type: VM >2014-08-11 13:38:02,508 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [190609a0] Correlation ID: 190609a0, Call Stack: null, Custom Event ID: -1, Message: Network Interface nic1 (VirtIO) was plugged to VM vm111. (User: admin) >2014-08-11 13:38:02,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [190609a0] Correlation ID: 6e09a6b1, Call Stack: null, Custom Event ID: -1, Message: Interface nic1 (VirtIO) was added to VM vm111. (User: admin) >2014-08-11 13:38:02,838 INFO [org.ovirt.engine.core.bll.network.vm.ReorderVmNicsCommand] (http--0.0.0.0-8080-3) [ec1b0ef] Running command: ReorderVmNicsCommand internal: false. Entities affected : ID: 2e77d03b-978a-41d9-a9e9-a43c7035366b Type: VMAction group CONFIGURE_VM_NETWORK with role type USER >2014-08-11 13:38:09,127 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (http--0.0.0.0-8080-4) [37572d88] Lock Acquired to object EngineLock [exclusiveLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM_DISK_BOOT >, sharedLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM >] >2014-08-11 13:38:09,159 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (http--0.0.0.0-8080-4) [37572d88] Running command: AddDiskCommand internal: false. Entities affected : ID: 2e77d03b-978a-41d9-a9e9-a43c7035366b Type: VMAction group CONFIGURE_VM_STORAGE with role type USER, ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 13:38:09,258 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (http--0.0.0.0-8080-4) [e61bec0] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-11 13:38:09,292 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (http--0.0.0.0-8080-4) [e61bec0] Lock freed to object EngineLock [exclusiveLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM_DISK_BOOT >, sharedLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM >] >2014-08-11 13:38:09,293 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (http--0.0.0.0-8080-4) [e61bec0] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 056639f7-5123-4146-b4f5-49c9284f7ff6, imageSizeInBytes = 1073741824, volumeFormat = RAW, newImageId = eda52843-98a1-4b68-b5ea-033ed979b96a, newImageDescription = ), log id: 6ac0fc34 >2014-08-11 13:38:09,295 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (http--0.0.0.0-8080-4) [e61bec0] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 13:38:09,679 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (http--0.0.0.0-8080-4) [e61bec0] FINISH, CreateImageVDSCommand, return: eda52843-98a1-4b68-b5ea-033ed979b96a, log id: 6ac0fc34 >2014-08-11 13:38:09,723 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (http--0.0.0.0-8080-4) [e61bec0] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 9b8db31f-376d-4b7d-ac35-b76718bc20ef >2014-08-11 13:38:09,724 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (http--0.0.0.0-8080-4) [e61bec0] CommandMultiAsyncTasks::AttachTask: Attaching task 5f1e5ee7-2cd0-46a4-a8ae-beda739f5701 to command 9b8db31f-376d-4b7d-ac35-b76718bc20ef. >2014-08-11 13:38:09,747 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (http--0.0.0.0-8080-4) [e61bec0] Adding task 5f1e5ee7-2cd0-46a4-a8ae-beda739f5701 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 13:38:09,822 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [e61bec0] Correlation ID: 37572d88, Job ID: bbd9ddbf-0baf-4021-8cae-7231c95a6129, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of vm111_Disk1 was initiated on VM vm111 by admin. >2014-08-11 13:38:09,823 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (http--0.0.0.0-8080-4) [e61bec0] BaseAsyncTask::startPollingTask: Starting to poll task 5f1e5ee7-2cd0-46a4-a8ae-beda739f5701. >2014-08-11 13:38:16,956 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-43) Setting new tasks map. The map contains now 1 tasks >2014-08-11 13:38:18,688 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-4) [7b1ee606] Lock Acquired to object EngineLock [exclusiveLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM >, sharedLocks= ] >2014-08-11 13:38:18,756 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [7b1ee606] START, IsVmDuringInitiatingVDSCommand( vmId = 2e77d03b-978a-41d9-a9e9-a43c7035366b), log id: 46e946a2 >2014-08-11 13:38:18,756 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [7b1ee606] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 46e946a2 >2014-08-11 13:38:18,758 WARN [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-4) [7b1ee606] CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_DISKS_LOCKED,$diskAliases vm111_Disk1 >2014-08-11 13:38:18,759 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-4) [7b1ee606] Lock freed to object EngineLock [exclusiveLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM >, sharedLocks= ] >2014-08-11 13:38:19,625 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-47) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-11 13:38:19,631 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-47) SPMAsyncTask::PollTask: Polling task 5f1e5ee7-2cd0-46a4-a8ae-beda739f5701 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 13:38:19,654 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-47) BaseAsyncTask::onTaskEndSuccess: Task 5f1e5ee7-2cd0-46a4-a8ae-beda739f5701 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 13:38:19,655 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-47) CommandAsyncTask::endActionIfNecessary: All tasks of command 9b8db31f-376d-4b7d-ac35-b76718bc20ef has ended -> executing endAction >2014-08-11 13:38:19,655 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-47) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 9b8db31f-376d-4b7d-ac35-b76718bc20ef): calling endAction . >2014-08-11 13:38:19,656 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddDisk, executionIndex: 0 >2014-08-11 13:38:19,670 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (org.ovirt.thread.pool-8-thread-34) [37572d88] Ending command successfully: org.ovirt.engine.core.bll.AddDiskCommand >2014-08-11 13:38:19,696 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-8-thread-34) [e61bec0] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-08-11 13:38:19,704 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-34) [e61bec0] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 056639f7-5123-4146-b4f5-49c9284f7ff6, imageId = eda52843-98a1-4b68-b5ea-033ed979b96a), log id: 7d96f931 >2014-08-11 13:38:19,731 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-34) [e61bec0] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@b86bc654, log id: 7d96f931 >2014-08-11 13:38:19,761 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-34) Correlation ID: 37572d88, Call Stack: null, Custom Event ID: -1, Message: The disk vm111_Disk1 was successfully added to VM vm111. >2014-08-11 13:38:19,762 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk completed, handling the result. >2014-08-11 13:38:19,762 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk succeeded, clearing tasks. >2014-08-11 13:38:19,770 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 5f1e5ee7-2cd0-46a4-a8ae-beda739f5701 >2014-08-11 13:38:19,770 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 5f1e5ee7-2cd0-46a4-a8ae-beda739f5701), log id: 17a6385e >2014-08-11 13:38:19,772 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=5f1e5ee7-2cd0-46a4-a8ae-beda739f5701), log id: 6df0b853 >2014-08-11 13:38:19,984 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) FINISH, HSMClearTaskVDSCommand, log id: 6df0b853 >2014-08-11 13:38:19,984 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) FINISH, SPMClearTaskVDSCommand, log id: 17a6385e >2014-08-11 13:38:19,993 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) BaseAsyncTask::removeTaskFromDB: Removed task 5f1e5ee7-2cd0-46a4-a8ae-beda739f5701 from DataBase >2014-08-11 13:38:19,994 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 9b8db31f-376d-4b7d-ac35-b76718bc20ef >2014-08-11 13:38:29,403 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-4) [2e465b9f] Lock Acquired to object EngineLock [exclusiveLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM >, sharedLocks= ] >2014-08-11 13:38:29,447 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [2e465b9f] START, IsVmDuringInitiatingVDSCommand( vmId = 2e77d03b-978a-41d9-a9e9-a43c7035366b), log id: 546753f4 >2014-08-11 13:38:29,448 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [2e465b9f] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 546753f4 >2014-08-11 13:38:29,536 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-1) [2e465b9f] Running command: RunVmCommand internal: false. Entities affected : ID: 2e77d03b-978a-41d9-a9e9-a43c7035366b Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 13:38:29,585 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-1) [2e465b9f] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@f7765a26), log id: 62b74c7f >2014-08-11 13:38:29,594 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-1) [2e465b9f] FINISH, UpdateVmDynamicDataVDSCommand, log id: 62b74c7f >2014-08-11 13:38:29,601 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-1) [2e465b9f] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b, vm=VM [vm111]), log id: 7542b675 >2014-08-11 13:38:29,684 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand] (org.ovirt.thread.pool-8-thread-1) [2e465b9f] START, CreateVmFromCloudInitVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b, vm=VM [vm111]), log id: 5d554bf7 >2014-08-11 13:38:29,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-1) [2e465b9f] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{shared=false, iface=ide, index=3, specParams={vmPayload={file={openstack/latest/meta_data.json=ewogICJsYXVuY2hfaW5kZXgiIDogIjAiLAogICJhdmFpbGFiaWxpdHlfem9uZSIgOiAibm92YSIs >CiAgIm5ldHdvcmstaW50ZXJmYWNlcyIgOiAiYXV0b1xuIiwKICAibmV0d29ya19jb25maWciIDog >ewogICAgImNvbnRlbnRfcGF0aCIgOiAiL2NvbnRlbnQvMDAwMCIsCiAgICAicGF0aCIgOiAiL2V0 >Yy9uZXR3b3JrL2ludGVyZmFjZXMiCiAgfSwKICAidXVpZCIgOiAiYWE1MTI4MjMtZTI1Zi00NjY5 >LThiNWQtMmEwNTFlNzU4OGIxIiwKICAibWV0YSIgOiB7CiAgICAiZXNzZW50aWFsIiA6ICJmYWxz >ZSIsCiAgICAicm9sZSIgOiAic2VydmVyIiwKICAgICJkc21vZGUiIDogImxvY2FsIgogIH0KfQ== >, openstack/latest/user_data=I2Nsb3VkLWNvbmZpZwpzc2hfcHdhdXRoOiB0cnVlCmRpc2FibGVfcm9vdDogMApvdXRwdXQ6CiAg >YWxsOiAnPj4gL3Zhci9sb2cvY2xvdWQtaW5pdC1vdXRwdXQubG9nJwpjaHBhc3N3ZDoKICBleHBp >cmU6IGZhbHNlCnJ1bmNtZDoKLSAnc2VkIC1pICcnL15kYXRhc291cmNlX2xpc3Q6IC9kJycgL2V0 >Yy9jbG91ZC9jbG91ZC5jZmc7IGVjaG8gJydkYXRhc291cmNlX2xpc3Q6CiAgWyJOb0Nsb3VkIiwg >IkNvbmZpZ0RyaXZlIl0nJyA+PiAvZXRjL2Nsb3VkL2Nsb3VkLmNmZycK >, openstack/content/0000=YXV0bwo= >}, volId=config-2}}, path=, device=cdrom, type=disk, readonly=true, deviceId=2cc9de31-f5af-450c-95d3-e216e1d03a7d}, {specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=6ee3a4eb-77a7-49b7-afc1-18d583950807}, {shared=false, iface=ide, index=2, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=5718e03c-a717-4b3f-b8cd-59430a59b07c}, {shared=false, volumeID=eda52843-98a1-4b68-b5ea-033ed979b96a, index=0, iface=virtio, imageID=056639f7-5123-4146-b4f5-49c9284f7ff6, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, specParams={}, optional=false, propagateErrors=off, device=disk, poolID=00000002-0002-0002-0002-00000000037f, format=raw, type=disk, readonly=false, deviceId=056639f7-5123-4146-b4f5-49c9284f7ff6}, {nicModel=pv, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:58, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=f22ea9ec-2eca-4d7f-a0cc-5d5d362e65df}, {index=0, model=virtio-scsi, specParams={}, device=scsi, type=controller, deviceId=37fe3c50-443a-46ce-aa63-7dc79581fa87}],vmName=vm111,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 13:38:29,756 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand] (org.ovirt.thread.pool-8-thread-1) [2e465b9f] FINISH, CreateVmFromCloudInitVDSCommand, log id: 5d554bf7 >2014-08-11 13:38:29,784 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-1) [2e465b9f] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 7542b675 >2014-08-11 13:38:29,785 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-1) [2e465b9f] Lock freed to object EngineLock [exclusiveLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM >, sharedLocks= ] >2014-08-11 13:38:29,801 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-1) [2e465b9f] Correlation ID: 2e465b9f, Job ID: db0449e8-b340-4e51-aaf8-9be6b07f4358, Call Stack: null, Custom Event ID: -1, Message: VM vm111 was started by admin (Host: h1). >2014-08-11 13:38:32,321 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) VM vm111 2e77d03b-978a-41d9-a9e9-a43c7035366b moved from WaitForLaunch --> PoweringUp >2014-08-11 13:38:32,322 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-84) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[2e77d03b-978a-41d9-a9e9-a43c7035366b]), log id: 11afb3c0 >2014-08-11 13:38:32,333 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-84) FINISH, FullListVdsCommand, return: [{displaySecurePort=5903, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.35.0.105, pauseCode=NOERR, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=2046, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@26420b52, vmName=vm111, fileTransferEnable=true, cpuType=SandyBridge}], log id: 11afb3c0 >2014-08-11 13:38:32,339 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Received a memballoon Device without an address when processing VM 2e77d03b-978a-41d9-a9e9-a43c7035366b devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-08-11 13:38:32,341 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Received a spice Device without an address when processing VM 2e77d03b-978a-41d9-a9e9-a43c7035366b devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5903, type=graphics} >2014-08-11 13:39:31,069 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) VM vm111 2e77d03b-978a-41d9-a9e9-a43c7035366b moved from PoweringUp --> Up >2014-08-11 13:39:31,133 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) Correlation ID: 2e465b9f, Job ID: db0449e8-b340-4e51-aaf8-9be6b07f4358, Call Stack: null, Custom Event ID: -1, Message: VM vm111 started on Host h1 >2014-08-11 13:39:46,959 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-91) Setting new tasks map. The map contains now 0 tasks >2014-08-11 13:39:46,960 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-91) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 13:41:14,095 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [292eb8da] Lock Acquired to object EngineLock [exclusiveLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM >, sharedLocks= ] >2014-08-11 13:41:14,220 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-42) [292eb8da] Running command: HibernateVmCommand internal: false. Entities affected : ID: 2e77d03b-978a-41d9-a9e9-a43c7035366b Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 13:41:14,228 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-42) [292eb8da] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@f930a119), log id: 67ead39d >2014-08-11 13:41:14,230 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-42) [292eb8da] FINISH, UpdateVmDynamicDataVDSCommand, log id: 67ead39d >2014-08-11 13:41:14,237 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-42) [292eb8da] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = bf2993d2-bff6-444c-9da9-d4c3a7597829, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = 5948021e-4b58-4f2a-ac41-9d0f1f403a84, newImageDescription = ), log id: 59a59352 >2014-08-11 13:41:14,238 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-42) [292eb8da] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 13:41:14,593 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-42) [292eb8da] FINISH, CreateImageVDSCommand, return: 5948021e-4b58-4f2a-ac41-9d0f1f403a84, log id: 59a59352 >2014-08-11 13:41:14,607 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-42) [292eb8da] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 41956c7b-9d30-4785-964a-2bfa02a035e0 >2014-08-11 13:41:14,608 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-42) [292eb8da] CommandMultiAsyncTasks::AttachTask: Attaching task 901ed003-f984-4b11-b0b1-e7d805e1abd9 to command 41956c7b-9d30-4785-964a-2bfa02a035e0. >2014-08-11 13:41:14,621 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-42) [292eb8da] Adding task 901ed003-f984-4b11-b0b1-e7d805e1abd9 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 13:41:14,642 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-42) [292eb8da] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 38ec7e8a-2161-4a69-b323-65bef8637062, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = 9acccb34-8ee8-4d05-ba08-83b1470c4b6a, newImageDescription = ), log id: 1911c232 >2014-08-11 13:41:14,644 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-42) [292eb8da] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 13:41:15,167 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-42) [292eb8da] FINISH, CreateImageVDSCommand, return: 9acccb34-8ee8-4d05-ba08-83b1470c4b6a, log id: 1911c232 >2014-08-11 13:41:15,289 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-42) [292eb8da] CommandMultiAsyncTasks::AttachTask: Attaching task d55473bd-1266-4dbf-a079-4955bf34a14c to command 41956c7b-9d30-4785-964a-2bfa02a035e0. >2014-08-11 13:41:15,304 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-42) [292eb8da] Adding task d55473bd-1266-4dbf-a079-4955bf34a14c (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 13:41:15,328 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-42) [292eb8da] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@3dce2174), log id: 5ccce627 >2014-08-11 13:41:15,337 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-42) [292eb8da] FINISH, UpdateVmDynamicDataVDSCommand, log id: 5ccce627 >2014-08-11 13:41:15,379 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-42) [292eb8da] Correlation ID: 292eb8da, Job ID: 8de3f57d-9b12-4867-b4be-8a42e5357303, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm111 was initiated by User admin (Host: h1). >2014-08-11 13:41:15,380 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-42) [292eb8da] BaseAsyncTask::startPollingTask: Starting to poll task 901ed003-f984-4b11-b0b1-e7d805e1abd9. >2014-08-11 13:41:15,381 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-42) [292eb8da] BaseAsyncTask::startPollingTask: Starting to poll task d55473bd-1266-4dbf-a079-4955bf34a14c. >2014-08-11 13:41:15,381 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-42) [292eb8da] Lock freed to object EngineLock [exclusiveLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM >, sharedLocks= ] >2014-08-11 13:41:19,659 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-57) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 13:41:19,666 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-57) SPMAsyncTask::PollTask: Polling task d55473bd-1266-4dbf-a079-4955bf34a14c (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 13:41:19,703 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-57) BaseAsyncTask::onTaskEndSuccess: Task d55473bd-1266-4dbf-a079-4955bf34a14c (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 13:41:19,704 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-57) Task with DB Task ID d1e8780f-d959-4a4a-a0f8-ed28d6784f23 and VDSM Task ID 901ed003-f984-4b11-b0b1-e7d805e1abd9 is in state Polling. End action for command 41956c7b-9d30-4785-964a-2bfa02a035e0 will proceed when all the entitys tasks are completed. >2014-08-11 13:41:19,704 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-57) SPMAsyncTask::PollTask: Polling task 901ed003-f984-4b11-b0b1-e7d805e1abd9 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 13:41:19,711 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-57) BaseAsyncTask::onTaskEndSuccess: Task 901ed003-f984-4b11-b0b1-e7d805e1abd9 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 13:41:19,712 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-57) CommandAsyncTask::endActionIfNecessary: All tasks of command 41956c7b-9d30-4785-964a-2bfa02a035e0 has ended -> executing endAction >2014-08-11 13:41:19,713 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-57) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 41956c7b-9d30-4785-964a-2bfa02a035e0): calling endAction . >2014-08-11 13:41:19,713 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-1) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-11 13:41:19,761 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-1) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-11 13:41:19,799 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-1) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,bf2993d2-bff6-444c-9da9-d4c3a7597829,5948021e-4b58-4f2a-ac41-9d0f1f403a84,38ec7e8a-2161-4a69-b323-65bef8637062,9acccb34-8ee8-4d05-ba08-83b1470c4b6a), log id: 1e22f98 >2014-08-11 13:41:19,801 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-1) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,bf2993d2-bff6-444c-9da9-d4c3a7597829,5948021e-4b58-4f2a-ac41-9d0f1f403a84,38ec7e8a-2161-4a69-b323-65bef8637062,9acccb34-8ee8-4d05-ba08-83b1470c4b6a), log id: 40e962d4 >2014-08-11 13:41:19,807 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-1) FINISH, HibernateBrokerVDSCommand, log id: 40e962d4 >2014-08-11 13:41:19,819 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-1) FINISH, HibernateVDSCommand, log id: 1e22f98 >2014-08-11 13:41:19,820 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-1) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-11 13:41:19,820 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-1) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-11 13:41:19,827 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-1) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 901ed003-f984-4b11-b0b1-e7d805e1abd9 >2014-08-11 13:41:19,828 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 901ed003-f984-4b11-b0b1-e7d805e1abd9), log id: 97d7648 >2014-08-11 13:41:19,829 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=901ed003-f984-4b11-b0b1-e7d805e1abd9), log id: be075d6 >2014-08-11 13:41:19,933 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) FINISH, HSMClearTaskVDSCommand, log id: be075d6 >2014-08-11 13:41:19,933 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) FINISH, SPMClearTaskVDSCommand, log id: 97d7648 >2014-08-11 13:41:19,943 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-1) BaseAsyncTask::removeTaskFromDB: Removed task 901ed003-f984-4b11-b0b1-e7d805e1abd9 from DataBase >2014-08-11 13:41:19,943 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-1) SPMAsyncTask::ClearAsyncTask: Attempting to clear task d55473bd-1266-4dbf-a079-4955bf34a14c >2014-08-11 13:41:19,944 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = d55473bd-1266-4dbf-a079-4955bf34a14c), log id: 6b4fd69b >2014-08-11 13:41:19,945 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=d55473bd-1266-4dbf-a079-4955bf34a14c), log id: 46929127 >2014-08-11 13:41:20,026 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) FINISH, HSMClearTaskVDSCommand, log id: 46929127 >2014-08-11 13:41:20,026 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) FINISH, SPMClearTaskVDSCommand, log id: 6b4fd69b >2014-08-11 13:41:20,042 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-1) BaseAsyncTask::removeTaskFromDB: Removed task d55473bd-1266-4dbf-a079-4955bf34a14c from DataBase >2014-08-11 13:41:20,043 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-1) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 41956c7b-9d30-4785-964a-2bfa02a035e0 >2014-08-11 13:41:20,929 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-54) START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b, force=false, secondsToWait=0, gracefully=false, reason=), log id: 1d18cda >2014-08-11 13:41:20,939 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-54) FINISH, DestroyVDSCommand, log id: 1d18cda >2014-08-11 13:41:20,978 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm111 on Host h1 is suspended. >2014-08-11 13:41:20,980 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) VM vm111 (2e77d03b-978a-41d9-a9e9-a43c7035366b) is running in db and not running in VDS h1 >2014-08-11 13:41:20,981 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-54) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[2e77d03b-978a-41d9-a9e9-a43c7035366b]), log id: 7e5aedd3 >2014-08-11 13:41:20,985 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-54) FINISH, FullListVdsCommand, return: [], log id: 7e5aedd3 >2014-08-11 13:41:21,090 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-7) [2d40f982] Running command: ProcessDownVmCommand internal: true. >2014-08-11 13:41:29,049 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-4) [80d2428] Lock Acquired to object EngineLock [exclusiveLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM >, sharedLocks= ] >2014-08-11 13:41:29,089 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [80d2428] START, IsVmDuringInitiatingVDSCommand( vmId = 2e77d03b-978a-41d9-a9e9-a43c7035366b), log id: 7099d63 >2014-08-11 13:41:29,090 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [80d2428] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 7099d63 >2014-08-11 13:41:29,187 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-21) [80d2428] Running command: RunVmCommand internal: false. Entities affected : ID: 2e77d03b-978a-41d9-a9e9-a43c7035366b Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 13:41:29,227 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-21) [80d2428] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@813ab654), log id: 6965f546 >2014-08-11 13:41:29,236 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-21) [80d2428] FINISH, UpdateVmDynamicDataVDSCommand, log id: 6965f546 >2014-08-11 13:41:29,240 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-21) [80d2428] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b, vm=VM [vm111]), log id: 15db699c >2014-08-11 13:41:29,243 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-21) [80d2428] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b, vm=VM [vm111]), log id: 410331f6 >2014-08-11 13:41:29,261 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-21) [80d2428] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,hiberVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,bf2993d2-bff6-444c-9da9-d4c3a7597829,5948021e-4b58-4f2a-ac41-9d0f1f403a84,38ec7e8a-2161-4a69-b323-65bef8637062,9acccb34-8ee8-4d05-ba08-83b1470c4b6a,transparentHugePages=true,vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=6ee3a4eb-77a7-49b7-afc1-18d583950807}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=5718e03c-a717-4b3f-b8cd-59430a59b07c}, {shared=false, index=0, volumeID=eda52843-98a1-4b68-b5ea-033ed979b96a, propagateErrors=off, format=raw, type=disk, iface=virtio, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, imageID=056639f7-5123-4146-b4f5-49c9284f7ff6, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=056639f7-5123-4146-b4f5-49c9284f7ff6}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:58, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=f22ea9ec-2eca-4d7f-a0cc-5d5d362e65df}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=37fe3c50-443a-46ce-aa63-7dc79581fa87}],vmName=vm111,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 13:41:29,325 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-21) [80d2428] FINISH, CreateVDSCommand, log id: 410331f6 >2014-08-11 13:41:29,344 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-21) [80d2428] FINISH, CreateVmVDSCommand, return: RestoringState, log id: 15db699c >2014-08-11 13:41:29,345 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-21) [80d2428] Lock freed to object EngineLock [exclusiveLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM >, sharedLocks= ] >2014-08-11 13:41:29,352 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-21) [80d2428] Correlation ID: 80d2428, Job ID: 03d8729c-b3c6-4f0c-8de4-ab5e089f331d, Call Stack: null, Custom Event ID: -1, Message: VM vm111 was started by admin (Host: h1). >2014-08-11 13:41:30,497 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) VM vm111 2e77d03b-978a-41d9-a9e9-a43c7035366b moved from RestoringState --> Up >2014-08-11 13:41:30,498 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-84) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[2e77d03b-978a-41d9-a9e9-a43c7035366b]), log id: 6195d9d0 >2014-08-11 13:41:30,509 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-84) FINISH, FullListVdsCommand, return: [{displaySecurePort=5903, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=2479, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@350f18bd, vmName=vm111, fileTransferEnable=true, cpuType=SandyBridge}], log id: 6195d9d0 >2014-08-11 13:41:30,512 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Received a memballoon Device without an address when processing VM 2e77d03b-978a-41d9-a9e9-a43c7035366b devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 13:41:30,514 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Received a spice Device without an address when processing VM 2e77d03b-978a-41d9-a9e9-a43c7035366b devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5903, type=graphics} >2014-08-11 13:41:30,611 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-84) Correlation ID: 80d2428, Job ID: 03d8729c-b3c6-4f0c-8de4-ab5e089f331d, Call Stack: null, Custom Event ID: -1, Message: VM vm111 started on Host h1 >2014-08-11 13:41:30,620 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (DefaultQuartzScheduler_Worker-84) [7c1be519] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-11 13:41:30,644 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-84) [7c1be519] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = bf2993d2-bff6-444c-9da9-d4c3a7597829, postZeros = false, forceDelete = false), log id: 2712e599 >2014-08-11 13:41:31,178 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-84) [7c1be519] FINISH, DeleteImageGroupVDSCommand, log id: 2712e599 >2014-08-11 13:41:31,287 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-84) [7c1be519] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command bca58c2d-4e04-4030-abdf-7f06ba89dc97 >2014-08-11 13:41:31,288 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-84) [7c1be519] CommandMultiAsyncTasks::AttachTask: Attaching task 4ba0b739-37a3-4598-a07a-1c42525cb689 to command bca58c2d-4e04-4030-abdf-7f06ba89dc97. >2014-08-11 13:41:31,307 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-84) [7c1be519] Adding task 4ba0b739-37a3-4598-a07a-1c42525cb689 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 13:41:31,332 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-84) [7c1be519] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 38ec7e8a-2161-4a69-b323-65bef8637062, postZeros = false, forceDelete = false), log id: 2f3f1f57 >2014-08-11 13:41:31,832 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-84) [7c1be519] FINISH, DeleteImageGroupVDSCommand, log id: 2f3f1f57 >2014-08-11 13:41:31,929 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-84) [7c1be519] CommandMultiAsyncTasks::AttachTask: Attaching task 289b21dd-4c41-439d-9fbd-7dfd0539f59c to command bca58c2d-4e04-4030-abdf-7f06ba89dc97. >2014-08-11 13:41:32,035 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-84) [7c1be519] Adding task 289b21dd-4c41-439d-9fbd-7dfd0539f59c (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 13:41:32,063 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-84) [7c1be519] BaseAsyncTask::startPollingTask: Starting to poll task 4ba0b739-37a3-4598-a07a-1c42525cb689. >2014-08-11 13:41:32,063 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-84) [7c1be519] BaseAsyncTask::startPollingTask: Starting to poll task 289b21dd-4c41-439d-9fbd-7dfd0539f59c. >2014-08-11 13:41:32,064 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-84) [7c1be519] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@33c0e9dd), log id: 31610d92 >2014-08-11 13:41:32,077 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-84) [7c1be519] FINISH, UpdateVmDynamicDataVDSCommand, log id: 31610d92 >2014-08-11 13:41:39,713 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-5) task id 4ba0b739-37a3-4598-a07a-1c42525cb689 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:41:39,714 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-5) task id 289b21dd-4c41-439d-9fbd-7dfd0539f59c is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:41:44,398 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-27) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[2e77d03b-978a-41d9-a9e9-a43c7035366b]), log id: 3d4adc6f >2014-08-11 13:41:44,412 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-27) FINISH, FullListVdsCommand, return: [{displaySecurePort=5903, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=2479, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@7dce1ad8, vmName=vm111, fileTransferEnable=true, cpuType=SandyBridge}], log id: 3d4adc6f >2014-08-11 13:41:44,424 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Received a memballoon Device without an address when processing VM 2e77d03b-978a-41d9-a9e9-a43c7035366b devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 13:41:44,426 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Received a spice Device without an address when processing VM 2e77d03b-978a-41d9-a9e9-a43c7035366b devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5903, type=graphics} >2014-08-11 13:41:48,860 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [40f97e44] Lock Acquired to object EngineLock [exclusiveLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM >, sharedLocks= ] >2014-08-11 13:42:11,596 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-32) task id 4ba0b739-37a3-4598-a07a-1c42525cb689 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:42:11,600 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-32) task id 289b21dd-4c41-439d-9fbd-7dfd0539f59c is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:42:19,880 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-57) Setting new tasks map. The map contains now 2 tasks >2014-08-11 13:42:22,005 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) task id 4ba0b739-37a3-4598-a07a-1c42525cb689 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:42:22,200 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) task id 289b21dd-4c41-439d-9fbd-7dfd0539f59c is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:42:22,607 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [40f97e44] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:258) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:739) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view6.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 13:42:25,804 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [40f97e44] Lock freed to object EngineLock [exclusiveLocks= key: 2e77d03b-978a-41d9-a9e9-a43c7035366b value: VM >, sharedLocks= ] >2014-08-11 13:42:32,619 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) task id 4ba0b739-37a3-4598-a07a-1c42525cb689 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:42:32,619 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) task id 4ba0b739-37a3-4598-a07a-1c42525cb689 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:42:32,620 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) task id 289b21dd-4c41-439d-9fbd-7dfd0539f59c has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:42:32,620 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-81) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 13:42:32,620 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) task id 4ba0b739-37a3-4598-a07a-1c42525cb689 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:42:32,621 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) task id 289b21dd-4c41-439d-9fbd-7dfd0539f59c has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:42:32,629 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) task id 4ba0b739-37a3-4598-a07a-1c42525cb689 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:42:32,629 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) SPMAsyncTask::PollTask: Polling task 4ba0b739-37a3-4598-a07a-1c42525cb689 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 13:42:32,643 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) BaseAsyncTask::onTaskEndSuccess: Task 4ba0b739-37a3-4598-a07a-1c42525cb689 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 13:42:32,644 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-81) Task with DB Task ID eb23ed74-29e0-4971-8303-c77410c01de1 and VDSM Task ID 289b21dd-4c41-439d-9fbd-7dfd0539f59c is in state Polling. End action for command bca58c2d-4e04-4030-abdf-7f06ba89dc97 will proceed when all the entitys tasks are completed. >2014-08-11 13:42:32,645 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) task id 289b21dd-4c41-439d-9fbd-7dfd0539f59c has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 13:42:32,645 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) SPMAsyncTask::PollTask: Polling task 289b21dd-4c41-439d-9fbd-7dfd0539f59c (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 13:42:32,660 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) BaseAsyncTask::onTaskEndSuccess: Task 289b21dd-4c41-439d-9fbd-7dfd0539f59c (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 13:42:32,660 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-81) CommandAsyncTask::endActionIfNecessary: All tasks of command bca58c2d-4e04-4030-abdf-7f06ba89dc97 has ended -> executing endAction >2014-08-11 13:42:32,661 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-81) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: bca58c2d-4e04-4030-abdf-7f06ba89dc97): calling endAction . >2014-08-11 13:42:32,661 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RunVm, executionIndex: 0 >2014-08-11 13:42:32,676 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-14) Ending command successfully: org.ovirt.engine.core.bll.RunVmCommand >2014-08-11 13:42:32,701 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm completed, handling the result. >2014-08-11 13:42:32,701 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm succeeded, clearing tasks. >2014-08-11 13:42:32,709 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 4ba0b739-37a3-4598-a07a-1c42525cb689 >2014-08-11 13:42:32,710 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 4ba0b739-37a3-4598-a07a-1c42525cb689), log id: 349fc328 >2014-08-11 13:42:32,711 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=4ba0b739-37a3-4598-a07a-1c42525cb689), log id: 64c59215 >2014-08-11 13:42:32,774 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, HSMClearTaskVDSCommand, log id: 64c59215 >2014-08-11 13:42:32,775 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, SPMClearTaskVDSCommand, log id: 349fc328 >2014-08-11 13:42:32,784 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) BaseAsyncTask::removeTaskFromDB: Removed task 4ba0b739-37a3-4598-a07a-1c42525cb689 from DataBase >2014-08-11 13:42:32,784 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 289b21dd-4c41-439d-9fbd-7dfd0539f59c >2014-08-11 13:42:32,785 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 289b21dd-4c41-439d-9fbd-7dfd0539f59c), log id: 17af409c >2014-08-11 13:42:32,787 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=289b21dd-4c41-439d-9fbd-7dfd0539f59c), log id: 72bacde9 >2014-08-11 13:42:32,841 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, HSMClearTaskVDSCommand, log id: 72bacde9 >2014-08-11 13:42:32,841 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, SPMClearTaskVDSCommand, log id: 17af409c >2014-08-11 13:42:32,850 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) BaseAsyncTask::removeTaskFromDB: Removed task 289b21dd-4c41-439d-9fbd-7dfd0539f59c from DataBase >2014-08-11 13:42:32,850 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity bca58c2d-4e04-4030-abdf-7f06ba89dc97 >2014-08-11 13:43:49,883 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-91) Setting new tasks map. The map contains now 0 tasks >2014-08-11 13:43:49,884 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-91) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 14:13:14,314 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-11 14:13:14,315 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-11 14:13:14,317 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-11 14:13:14,319 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-11 14:13:14,320 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-11 14:13:14,321 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-11 14:13:14,322 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-11 14:13:14,323 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-11 14:13:14,324 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-11 14:13:14,324 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-11 14:13:14,325 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-11 14:13:14,325 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-11 14:13:14,326 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-11 14:13:14,326 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-11 14:13:14,327 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-11 14:13:14,327 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-11 14:13:14,328 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-11 14:13:14,329 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-11 14:13:14,329 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-11 14:13:14,329 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-11 14:13:14,330 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-11 14:13:14,330 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-11 14:13:14,331 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_USER" is "engine". >2014-08-11 14:13:14,332 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-11 14:13:14,333 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-11 14:13:14,334 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-11 14:13:14,335 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-11 14:13:14,337 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-11 14:13:14,337 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-11 14:13:14,338 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-11 14:13:14,338 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-11 14:13:14,339 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-11 14:13:14,339 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-11 14:13:14,340 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-11 14:13:14,340 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-11 14:13:14,341 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-11 14:13:14,341 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-11 14:13:14,342 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-11 14:13:14,342 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-11 14:13:14,343 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-11 14:13:14,343 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-11 14:13:14,347 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-11 14:13:14,347 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-11 14:13:14,348 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-11 14:13:14,348 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-11 14:13:14,349 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-11 14:13:14,350 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-11 14:13:14,350 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-11 14:13:14,351 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-11 14:13:14,351 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-11 14:13:14,352 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-11 14:13:14,359 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-11 14:13:14,360 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-11 14:13:14,360 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-11 14:13:14,363 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-11 14:13:14,364 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-11 14:13:14,365 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-11 14:13:14,366 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-11 14:13:14,367 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-11 14:13:14,368 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-11 14:13:14,368 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-11 14:13:14,369 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-11 14:13:14,369 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-11 14:13:14,370 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_USER" is "ovirt". >2014-08-11 14:13:14,370 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-11 14:13:14,371 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-11 14:13:14,371 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-11 14:13:14,372 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-11 14:13:14,372 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-11 14:13:14,373 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-11 14:13:14,720 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-6) Start initializing Backend >2014-08-11 14:13:14,840 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-6) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 14:13:14,956 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-6) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 14:13:14,995 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-6) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-11 14:13:15,013 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-6) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-11 14:13:15,156 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-6) Running ovirt-engine 3.5.0.0 >2014-08-11 14:13:15,157 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-6) Start initializing dictionaries >2014-08-11 14:13:15,159 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-6) Finished initializing dictionaries >2014-08-11 14:13:15,160 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Start initializing AuditLogCleanupManager >2014-08-11 14:13:15,160 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-11 14:13:15,173 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Finished initializing AuditLogCleanupManager >2014-08-11 14:13:15,173 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Start initializing CommandEntityCleanupManager >2014-08-11 14:13:15,174 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-11 14:13:15,175 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-6) Finished initializing CommandEntityCleanupManager >2014-08-11 14:13:15,177 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-6) Start initializing TagsDirector >2014-08-11 14:13:15,189 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-6) Tag root added to tree >2014-08-11 14:13:15,198 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-6) Finished initializing TagsDirector >2014-08-11 14:13:15,199 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-6) Start initializing IsoDomainListSyncronizer >2014-08-11 14:13:15,205 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-6) Finished initializing IsoDomainListSyncronizer >2014-08-11 14:13:15,273 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-6) Completed initializing handlers >2014-08-11 14:13:15,295 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Start initializing ErrorTranslatorImpl >2014-08-11 14:13:15,310 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-11 14:13:15,311 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Finished initializing ErrorTranslatorImpl >2014-08-11 14:13:15,312 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Start initializing ErrorTranslatorImpl >2014-08-11 14:13:15,312 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-6) Finished initializing ErrorTranslatorImpl >2014-08-11 14:13:15,313 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-6) Mark incomplete jobs as UNKNOWN >2014-08-11 14:13:15,338 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-6) Start initializing JobRepositoryCleanupManager >2014-08-11 14:13:15,338 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-6) Finished initializing JobRepositoryCleanupManager >2014-08-11 14:13:15,339 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-6) Start initializing AutoRecoveryManager >2014-08-11 14:13:15,340 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-6) Finished initializing AutoRecoveryManager >2014-08-11 14:13:15,342 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-6) Start initializing ExecutionMessageDirector >2014-08-11 14:13:15,344 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-6) Finished initializing ExecutionMessageDirector >2014-08-11 14:13:15,383 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-6) Start of enabled extensions list >2014-08-11 14:13:15,384 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-6) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-11 14:13:15,385 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-6) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-11 14:13:15,386 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-6) End of enabled extensions list >2014-08-11 14:13:15,452 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-5) Start initializing DbUserCacheManager >2014-08-11 14:13:15,454 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-5) Finished initializing DbUserCacheManager >2014-08-11 14:13:15,469 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-5) Initialization of AsyncTaskManager completed successfully. >2014-08-11 14:13:15,471 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-5) Start initializing ResourceManager >2014-08-11 14:13:15,508 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-11 14:13:15,558 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-5) Entered VdsManager constructor >2014-08-11 14:13:15,592 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-5) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-11 14:13:15,685 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-5) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-11 14:13:15,686 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-5) Entered VdsManager constructor >2014-08-11 14:13:15,700 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-5) Initialize vdsBroker (galactica,54,321) >2014-08-11 14:13:15,729 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-5) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-11 14:13:15,752 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-5) Finished initializing ResourceManager >2014-08-11 14:13:15,757 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-5) Initialization of OvfDataUpdater completed successfully. >2014-08-11 14:13:15,758 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) Start scheduling to enable vds load balancer >2014-08-11 14:13:15,759 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) Finished scheduling to enable vds load balancer >2014-08-11 14:13:15,759 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) Start HA Reservation check >2014-08-11 14:13:15,760 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) Finished HA Reservation check >2014-08-11 14:13:15,769 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-5) Init VM custom properties utilities >2014-08-11 14:13:15,772 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-5) Init device custom properties utilities >2014-08-11 14:13:15,784 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) Initializing Scheduling manager >2014-08-11 14:13:15,815 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-11 14:13:15,816 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-11 14:13:15,817 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-11 14:13:15,824 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) External scheduler disabled, discovery skipped >2014-08-11 14:13:15,824 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) Initialized Scheduling manager >2014-08-11 14:13:15,825 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-5) Initializing DWH Heart Beat >2014-08-11 14:13:15,826 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-5) DWH Heart Beat initialized >2014-08-11 14:13:18,746 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: h2 >2014-08-11 14:13:19,300 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 4b4bfa67 >2014-08-11 14:13:19,313 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 4b4bfa67 >2014-08-11 14:13:19,317 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-11 14:13:19,321 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: UNASSIGNED not exist in string table >2014-08-11 14:13:19,321 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-11 14:13:19,322 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-11 14:13:19,323 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-11 14:13:19,323 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-11 14:13:19,324 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-11 14:13:19,324 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-11 14:13:19,325 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-11 14:13:19,326 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-11 14:13:19,327 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-11 14:13:19,327 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-11 14:13:19,329 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STOPPED not exist in string table >2014-08-11 14:13:19,329 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STARTED not exist in string table >2014-08-11 14:13:19,330 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_ERROR not exist in string table >2014-08-11 14:13:19,331 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-11 14:13:19,332 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-11 14:13:19,333 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-11 14:13:19,333 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-11 14:13:19,525 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-11 14:13:19,526 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is running with disabled SELinux. >2014-08-11 14:13:19,837 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h1 >2014-08-11 14:13:19,925 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [36ad5afc] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-11 14:13:25,018 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-2) Running command: LoginAdminUserCommand internal: false. >2014-08-11 14:13:25,041 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-11 14:13:25,777 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-23) [12ddcf92] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-11 14:13:25,812 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [12ddcf92] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-11 14:13:25,823 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [12ddcf92] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-11 14:13:25,902 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [12ddcf92] Correlation ID: 12ddcf92, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-11 14:13:26,052 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) [12ddcf92] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-11 14:13:26,076 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) [12ddcf92] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-11 14:13:26,118 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [12ddcf92] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-11 14:13:26,183 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-14) [12ddcf92] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 24a77785 >2014-08-11 14:13:26,366 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [12ddcf92] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 52 GB of free space >2014-08-11 14:13:26,415 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [12ddcf92] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 52 GB of free space >2014-08-11 14:13:26,418 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-14) [12ddcf92] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-11 14:13:26,442 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-14) [12ddcf92] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 16b9724a >2014-08-11 14:13:26,468 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-14) [12ddcf92] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 16b9724a >2014-08-11 14:13:26,469 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-14) [12ddcf92] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 24a77785 >2014-08-11 14:13:26,470 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-14) [12ddcf92] Discovered no tasks on Storage Pool Default >2014-08-11 14:13:28,954 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-3) Running command: LoginUserCommand internal: false. >2014-08-11 15:13:15,535 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-30) Principal admin::internal synchronized >2014-08-11 15:13:15,758 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VMs/Templates Ovf. >2014-08-11 15:13:16,083 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-37) [5a048568] Lock Acquired to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-11 15:13:16,087 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-37) [5a048568] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-11 15:13:16,149 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [5a048568] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 28e4c2fc >2014-08-11 15:13:16,150 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [5a048568] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-11 15:13:16,151 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [5a048568] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Sun Aug 03 23:07:30 IDT 2014"} > >2014-08-11 15:13:16,286 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [5a048568] FINISH, SetVolumeDescriptionVDSCommand, log id: 28e4c2fc >2014-08-11 15:13:16,459 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-11 15:13:16,467 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 3b13395c >2014-08-11 15:13:16,467 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] -- executeVdsBrokerCommand: >2014-08-11 15:13:16,468 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > size=40,960 > >2014-08-11 15:13:17,211 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] FINISH, UploadStreamVDSCommand, log id: 3b13395c >2014-08-11 15:13:17,269 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-37) [4257876b] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command b09b03cf-1449-44b2-aac6-298deefe6020 >2014-08-11 15:13:17,271 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-37) [4257876b] CommandMultiAsyncTasks::AttachTask: Attaching task 4a0bcdae-5e73-4e20-a595-442f6c6afd57 to command b09b03cf-1449-44b2-aac6-298deefe6020. >2014-08-11 15:13:17,303 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-37) [4257876b] Adding task 4a0bcdae-5e73-4e20-a595-442f6c6afd57 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:13:17,345 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-11 15:13:17,353 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 16f7ac48 >2014-08-11 15:13:17,354 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-11 15:13:17,355 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Mon Aug 11 15:13:16 IDT 2014","Size":40960} > >2014-08-11 15:13:17,467 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] FINISH, SetVolumeDescriptionVDSCommand, log id: 16f7ac48 >2014-08-11 15:13:17,480 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 5057e8a6 >2014-08-11 15:13:17,481 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-11 15:13:17,481 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Sun Aug 03 23:07:30 IDT 2014"} > >2014-08-11 15:13:17,603 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [4257876b] FINISH, SetVolumeDescriptionVDSCommand, log id: 5057e8a6 >2014-08-11 15:13:17,643 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-37) [3dfc10] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-11 15:13:17,645 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-37) [3dfc10] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 21159d90 >2014-08-11 15:13:17,645 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-37) [3dfc10] -- executeVdsBrokerCommand: >2014-08-11 15:13:17,645 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-37) [3dfc10] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > size=40,960 > >2014-08-11 15:13:18,450 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-37) [3dfc10] FINISH, UploadStreamVDSCommand, log id: 21159d90 >2014-08-11 15:13:18,453 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-37) [3dfc10] CommandMultiAsyncTasks::AttachTask: Attaching task ac87e95a-9367-45a0-b118-370607693fda to command b09b03cf-1449-44b2-aac6-298deefe6020. >2014-08-11 15:13:18,595 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-37) [3dfc10] Adding task ac87e95a-9367-45a0-b118-370607693fda (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:13:18,611 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-37) [3dfc10] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-11 15:13:18,612 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-37) [3dfc10] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-11 15:13:18,612 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-37) [3dfc10] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-11 15:13:18,613 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-37) [3dfc10] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-11 15:13:18,613 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-37) [3dfc10] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-11 15:13:18,616 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [3dfc10] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 49ce25a0 >2014-08-11 15:13:18,617 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [3dfc10] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-11 15:13:18,617 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [3dfc10] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Mon Aug 11 15:13:16 IDT 2014","Size":40960} > >2014-08-11 15:13:18,700 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-37) [3dfc10] FINISH, SetVolumeDescriptionVDSCommand, log id: 49ce25a0 >2014-08-11 15:13:18,711 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-37) [3dfc10] BaseAsyncTask::startPollingTask: Starting to poll task 4a0bcdae-5e73-4e20-a595-442f6c6afd57. >2014-08-11 15:13:18,711 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-37) [3dfc10] BaseAsyncTask::startPollingTask: Starting to poll task ac87e95a-9367-45a0-b118-370607693fda. >2014-08-11 15:13:25,556 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-77) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 15:13:25,565 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-77) SPMAsyncTask::PollTask: Polling task ac87e95a-9367-45a0-b118-370607693fda (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:13:25,566 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-77) BaseAsyncTask::onTaskEndSuccess: Task ac87e95a-9367-45a0-b118-370607693fda (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:13:25,567 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-77) Task with DB Task ID d9781556-c40b-4e90-a788-1962e3add811 and VDSM Task ID 4a0bcdae-5e73-4e20-a595-442f6c6afd57 is in state Polling. End action for command b09b03cf-1449-44b2-aac6-298deefe6020 will proceed when all the entitys tasks are completed. >2014-08-11 15:13:25,567 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-77) SPMAsyncTask::PollTask: Polling task 4a0bcdae-5e73-4e20-a595-442f6c6afd57 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:13:25,568 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-77) BaseAsyncTask::onTaskEndSuccess: Task 4a0bcdae-5e73-4e20-a595-442f6c6afd57 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:13:25,569 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-77) CommandAsyncTask::endActionIfNecessary: All tasks of command b09b03cf-1449-44b2-aac6-298deefe6020 has ended -> executing endAction >2014-08-11 15:13:25,569 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-77) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: b09b03cf-1449-44b2-aac6-298deefe6020): calling endAction . >2014-08-11 15:13:25,571 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-08-11 15:13:25,584 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-46) [5a048568] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-08-11 15:13:25,585 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-46) [5a048568] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-11 15:13:25,586 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-46) [5a048568] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-11 15:13:25,587 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-46) [5a048568] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-11 15:13:25,587 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-46) [5a048568] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-11 15:13:25,587 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-46) [5a048568] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-11 15:13:25,588 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [5a048568] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-08-11 15:13:25,589 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [5a048568] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-08-11 15:13:25,589 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [5a048568] SPMAsyncTask::ClearAsyncTask: Attempting to clear task ac87e95a-9367-45a0-b118-370607693fda >2014-08-11 15:13:25,591 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [5a048568] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = ac87e95a-9367-45a0-b118-370607693fda), log id: 732a18e2 >2014-08-11 15:13:25,594 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [5a048568] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=ac87e95a-9367-45a0-b118-370607693fda), log id: 6e921979 >2014-08-11 15:13:25,652 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [5a048568] FINISH, HSMClearTaskVDSCommand, log id: 6e921979 >2014-08-11 15:13:25,652 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [5a048568] FINISH, SPMClearTaskVDSCommand, log id: 732a18e2 >2014-08-11 15:13:25,702 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [5a048568] BaseAsyncTask::removeTaskFromDB: Removed task ac87e95a-9367-45a0-b118-370607693fda from DataBase >2014-08-11 15:13:25,702 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [5a048568] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 4a0bcdae-5e73-4e20-a595-442f6c6afd57 >2014-08-11 15:13:25,703 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [5a048568] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 4a0bcdae-5e73-4e20-a595-442f6c6afd57), log id: 188601ec >2014-08-11 15:13:25,704 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [5a048568] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=4a0bcdae-5e73-4e20-a595-442f6c6afd57), log id: 5e83988 >2014-08-11 15:13:25,771 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [5a048568] FINISH, HSMClearTaskVDSCommand, log id: 5e83988 >2014-08-11 15:13:25,771 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-46) [5a048568] FINISH, SPMClearTaskVDSCommand, log id: 188601ec >2014-08-11 15:13:25,785 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) [5a048568] BaseAsyncTask::removeTaskFromDB: Removed task 4a0bcdae-5e73-4e20-a595-442f6c6afd57 from DataBase >2014-08-11 15:13:25,785 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [5a048568] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity b09b03cf-1449-44b2-aac6-298deefe6020 >2014-08-11 15:13:39,594 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [1531827a] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 15:13:39,807 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-15) [1531827a] Running command: HibernateVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 15:13:39,848 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-15) [1531827a] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@71f0240c), log id: 47800490 >2014-08-11 15:13:39,850 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-15) [1531827a] FINISH, UpdateVmDynamicDataVDSCommand, log id: 47800490 >2014-08-11 15:13:39,867 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-15) [1531827a] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = cd2b567f-b105-4c1f-a388-c96d516877bc, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = 0797b137-8d0a-4c7c-9e6b-963f21c5797f, newImageDescription = ), log id: 72d0e0a4 >2014-08-11 15:13:39,873 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-15) [1531827a] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 15:13:40,473 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-15) [1531827a] FINISH, CreateImageVDSCommand, return: 0797b137-8d0a-4c7c-9e6b-963f21c5797f, log id: 72d0e0a4 >2014-08-11 15:13:40,516 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-15) [1531827a] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 1880e306-d49e-4ea7-859f-596ee98d0077 >2014-08-11 15:13:40,517 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-15) [1531827a] CommandMultiAsyncTasks::AttachTask: Attaching task b62f0f6c-7613-42ee-ae47-8a7115f3459e to command 1880e306-d49e-4ea7-859f-596ee98d0077. >2014-08-11 15:13:40,529 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-15) [1531827a] Adding task b62f0f6c-7613-42ee-ae47-8a7115f3459e (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:13:40,544 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-15) [1531827a] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 5108758f-8580-4eda-8a6f-b5b61e5e6d91, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = ace771d6-aeae-4505-b9de-eb22b38f8a98, newImageDescription = ), log id: 1422af90 >2014-08-11 15:13:40,547 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-15) [1531827a] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 15:13:41,264 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-15) [1531827a] FINISH, CreateImageVDSCommand, return: ace771d6-aeae-4505-b9de-eb22b38f8a98, log id: 1422af90 >2014-08-11 15:13:41,324 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-15) [1531827a] CommandMultiAsyncTasks::AttachTask: Attaching task 76fa20bc-68ba-4cf3-844c-34adb9c089a9 to command 1880e306-d49e-4ea7-859f-596ee98d0077. >2014-08-11 15:13:41,339 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-15) [1531827a] Adding task 76fa20bc-68ba-4cf3-844c-34adb9c089a9 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:13:41,349 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-15) [1531827a] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@a8d21c79), log id: f23d39e >2014-08-11 15:13:41,356 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-15) [1531827a] FINISH, UpdateVmDynamicDataVDSCommand, log id: f23d39e >2014-08-11 15:13:41,418 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-15) [1531827a] Correlation ID: 1531827a, Job ID: 2c0cfb54-d63e-4cad-a0df-9c2e70d28d69, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm was initiated by User admin (Host: h1). >2014-08-11 15:13:41,418 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-15) [1531827a] BaseAsyncTask::startPollingTask: Starting to poll task b62f0f6c-7613-42ee-ae47-8a7115f3459e. >2014-08-11 15:13:41,418 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-15) [1531827a] BaseAsyncTask::startPollingTask: Starting to poll task 76fa20bc-68ba-4cf3-844c-34adb9c089a9. >2014-08-11 15:13:41,419 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-15) [1531827a] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 15:13:45,570 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-22) Polling and updating Async Tasks: 4 tasks, 2 tasks to poll now >2014-08-11 15:13:45,579 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) SPMAsyncTask::PollTask: Polling task 76fa20bc-68ba-4cf3-844c-34adb9c089a9 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:13:45,630 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) BaseAsyncTask::onTaskEndSuccess: Task 76fa20bc-68ba-4cf3-844c-34adb9c089a9 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:13:45,630 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-22) Task with DB Task ID af96f52c-54ba-40a1-b6b9-18760e8f8484 and VDSM Task ID b62f0f6c-7613-42ee-ae47-8a7115f3459e is in state Polling. End action for command 1880e306-d49e-4ea7-859f-596ee98d0077 will proceed when all the entitys tasks are completed. >2014-08-11 15:13:45,631 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) SPMAsyncTask::PollTask: Polling task b62f0f6c-7613-42ee-ae47-8a7115f3459e (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:13:45,638 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) BaseAsyncTask::onTaskEndSuccess: Task b62f0f6c-7613-42ee-ae47-8a7115f3459e (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:13:45,639 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-22) CommandAsyncTask::endActionIfNecessary: All tasks of command 1880e306-d49e-4ea7-859f-596ee98d0077 has ended -> executing endAction >2014-08-11 15:13:45,639 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-22) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 1880e306-d49e-4ea7-859f-596ee98d0077): calling endAction . >2014-08-11 15:13:45,640 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-21) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-11 15:13:45,688 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-21) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-11 15:13:45,690 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-21) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,cd2b567f-b105-4c1f-a388-c96d516877bc,0797b137-8d0a-4c7c-9e6b-963f21c5797f,5108758f-8580-4eda-8a6f-b5b61e5e6d91,ace771d6-aeae-4505-b9de-eb22b38f8a98), log id: 13d67f47 >2014-08-11 15:13:45,692 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-21) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,cd2b567f-b105-4c1f-a388-c96d516877bc,0797b137-8d0a-4c7c-9e6b-963f21c5797f,5108758f-8580-4eda-8a6f-b5b61e5e6d91,ace771d6-aeae-4505-b9de-eb22b38f8a98), log id: 6e883a2c >2014-08-11 15:13:45,698 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-21) FINISH, HibernateBrokerVDSCommand, log id: 6e883a2c >2014-08-11 15:13:45,712 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-21) FINISH, HibernateVDSCommand, log id: 13d67f47 >2014-08-11 15:13:45,713 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-21) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-11 15:13:45,714 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-21) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-11 15:13:45,729 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-21) SPMAsyncTask::ClearAsyncTask: Attempting to clear task b62f0f6c-7613-42ee-ae47-8a7115f3459e >2014-08-11 15:13:45,730 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-21) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = b62f0f6c-7613-42ee-ae47-8a7115f3459e), log id: a46becf >2014-08-11 15:13:45,731 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-21) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=b62f0f6c-7613-42ee-ae47-8a7115f3459e), log id: 263891bb >2014-08-11 15:13:45,839 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-21) FINISH, HSMClearTaskVDSCommand, log id: 263891bb >2014-08-11 15:13:45,840 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-21) FINISH, SPMClearTaskVDSCommand, log id: a46becf >2014-08-11 15:13:45,853 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-21) BaseAsyncTask::removeTaskFromDB: Removed task b62f0f6c-7613-42ee-ae47-8a7115f3459e from DataBase >2014-08-11 15:13:45,854 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-21) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 76fa20bc-68ba-4cf3-844c-34adb9c089a9 >2014-08-11 15:13:45,854 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-21) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 76fa20bc-68ba-4cf3-844c-34adb9c089a9), log id: 375746f6 >2014-08-11 15:13:45,855 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-21) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=76fa20bc-68ba-4cf3-844c-34adb9c089a9), log id: 42f3dcdc >2014-08-11 15:13:46,057 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-21) FINISH, HSMClearTaskVDSCommand, log id: 42f3dcdc >2014-08-11 15:13:46,057 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-21) FINISH, SPMClearTaskVDSCommand, log id: 375746f6 >2014-08-11 15:13:46,068 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-21) BaseAsyncTask::removeTaskFromDB: Removed task 76fa20bc-68ba-4cf3-844c-34adb9c089a9 from DataBase >2014-08-11 15:13:46,069 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-21) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 1880e306-d49e-4ea7-859f-596ee98d0077 >2014-08-11 15:13:49,671 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-38) START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, force=false, secondsToWait=0, gracefully=false, reason=), log id: 38a674dc >2014-08-11 15:13:49,682 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-38) FINISH, DestroyVDSCommand, log id: 38a674dc >2014-08-11 15:13:49,746 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-38) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm on Host h1 is suspended. >2014-08-11 15:13:49,750 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) VM vm (14c0fa91-1814-438b-8d7a-4e7de1e1da25) is running in db and not running in VDS h1 >2014-08-11 15:13:49,752 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-38) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 22976b20 >2014-08-11 15:13:49,758 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-38) FINISH, FullListVdsCommand, return: [], log id: 22976b20 >2014-08-11 15:13:49,867 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-32) [7e280edb] Running command: ProcessDownVmCommand internal: true. >2014-08-11 15:14:01,758 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [4df050e5] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 15:14:01,845 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [4df050e5] START, IsVmDuringInitiatingVDSCommand( vmId = 14c0fa91-1814-438b-8d7a-4e7de1e1da25), log id: a2a62c0 >2014-08-11 15:14:01,845 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [4df050e5] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: a2a62c0 >2014-08-11 15:14:01,975 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-2) [4df050e5] Running command: RunVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 15:14:02,018 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-2) [4df050e5] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@75f571ca), log id: 329622d5 >2014-08-11 15:14:02,030 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-2) [4df050e5] FINISH, UpdateVmDynamicDataVDSCommand, log id: 329622d5 >2014-08-11 15:14:02,047 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-2) [4df050e5] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, vm=VM [vm]), log id: 5c428ee1 >2014-08-11 15:14:02,055 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-2) [4df050e5] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, vm=VM [vm]), log id: 7a1570fa >2014-08-11 15:14:02,099 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-2) [4df050e5] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,hiberVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,cd2b567f-b105-4c1f-a388-c96d516877bc,0797b137-8d0a-4c7c-9e6b-963f21c5797f,5108758f-8580-4eda-8a6f-b5b61e5e6d91,ace771d6-aeae-4505-b9de-eb22b38f8a98,transparentHugePages=true,vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=68e21ff2-4b5a-4b29-8765-eec79f1b15dd}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=496deb19-46c7-4b58-94a3-06b7fa34f7ac}, {shared=false, index=0, volumeID=3f8b5bbe-741b-4a4f-9397-84123d530772, propagateErrors=off, format=raw, type=disk, iface=virtio, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, imageID=99e5cda5-262b-4469-b336-04e3f0cf7c38, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=99e5cda5-262b-4469-b336-04e3f0cf7c38}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:57, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=1e5b18b9-16eb-4b96-83b6-4568b8bed355}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=3959560b-27d1-4acf-acd9-c39f0c4148b2}],vmName=vm,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 15:14:02,189 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-2) [4df050e5] FINISH, CreateVDSCommand, log id: 7a1570fa >2014-08-11 15:14:02,212 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-2) [4df050e5] FINISH, CreateVmVDSCommand, return: RestoringState, log id: 5c428ee1 >2014-08-11 15:14:02,213 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-2) [4df050e5] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 15:14:02,229 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-2) [4df050e5] Correlation ID: 4df050e5, Job ID: 37bd0363-8114-4610-bda6-cb6652999cab, Call Stack: null, Custom Event ID: -1, Message: VM vm was started by admin (Host: h1). >2014-08-11 15:14:05,000 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) VM vm 14c0fa91-1814-438b-8d7a-4e7de1e1da25 moved from RestoringState --> Up >2014-08-11 15:14:05,003 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-79) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 3c31adab >2014-08-11 15:14:05,014 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-79) FINISH, FullListVdsCommand, return: [{displaySecurePort=5902, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=10054, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@5983bc82, vmName=vm, fileTransferEnable=true, cpuType=SandyBridge}], log id: 3c31adab >2014-08-11 15:14:05,019 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) Received a memballoon Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 15:14:05,020 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) Received a spice Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5902, type=graphics} >2014-08-11 15:14:05,227 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-79) Correlation ID: 4df050e5, Job ID: 37bd0363-8114-4610-bda6-cb6652999cab, Call Stack: null, Custom Event ID: -1, Message: VM vm started on Host h1 >2014-08-11 15:14:05,249 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (DefaultQuartzScheduler_Worker-79) [744a0844] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-11 15:14:05,319 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-79) [744a0844] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = cd2b567f-b105-4c1f-a388-c96d516877bc, postZeros = false, forceDelete = false), log id: 145f0849 >2014-08-11 15:14:06,156 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-79) [744a0844] FINISH, DeleteImageGroupVDSCommand, log id: 145f0849 >2014-08-11 15:14:06,191 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-79) [744a0844] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command bc76d541-1202-4e12-9294-0e93b0b6598a >2014-08-11 15:14:06,191 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-79) [744a0844] CommandMultiAsyncTasks::AttachTask: Attaching task 3b93fb1e-cd32-47f0-95e9-22fc4512675d to command bc76d541-1202-4e12-9294-0e93b0b6598a. >2014-08-11 15:14:06,213 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-79) [744a0844] Adding task 3b93fb1e-cd32-47f0-95e9-22fc4512675d (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:14:06,238 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-79) [744a0844] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 5108758f-8580-4eda-8a6f-b5b61e5e6d91, postZeros = false, forceDelete = false), log id: d7c454 >2014-08-11 15:14:06,864 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-79) [744a0844] FINISH, DeleteImageGroupVDSCommand, log id: d7c454 >2014-08-11 15:14:06,917 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-79) [744a0844] CommandMultiAsyncTasks::AttachTask: Attaching task f7992afe-e86e-45cc-8d76-3177d5daead7 to command bc76d541-1202-4e12-9294-0e93b0b6598a. >2014-08-11 15:14:06,933 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-79) [744a0844] Adding task f7992afe-e86e-45cc-8d76-3177d5daead7 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:14:06,963 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) [744a0844] BaseAsyncTask::startPollingTask: Starting to poll task 3b93fb1e-cd32-47f0-95e9-22fc4512675d. >2014-08-11 15:14:06,963 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) [744a0844] BaseAsyncTask::startPollingTask: Starting to poll task f7992afe-e86e-45cc-8d76-3177d5daead7. >2014-08-11 15:14:06,965 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-79) [744a0844] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@6092c891), log id: 65ace479 >2014-08-11 15:14:06,975 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-79) [744a0844] FINISH, UpdateVmDynamicDataVDSCommand, log id: 65ace479 >2014-08-11 15:14:15,640 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-8) task id f7992afe-e86e-45cc-8d76-3177d5daead7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:14:15,641 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-8) task id 3b93fb1e-cd32-47f0-95e9-22fc4512675d is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:14:22,069 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-35) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 5cc6b6d9 >2014-08-11 15:14:22,079 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-35) FINISH, FullListVdsCommand, return: [{displaySecurePort=5902, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=10054, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@7a0e9024, vmName=vm, fileTransferEnable=true, cpuType=SandyBridge}], log id: 5cc6b6d9 >2014-08-11 15:14:22,084 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Received a memballoon Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 15:14:22,086 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Received a spice Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5902, type=graphics} >2014-08-11 15:14:25,643 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-38) task id f7992afe-e86e-45cc-8d76-3177d5daead7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:14:25,644 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-38) task id 3b93fb1e-cd32-47f0-95e9-22fc4512675d is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:14:35,647 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-69) task id f7992afe-e86e-45cc-8d76-3177d5daead7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:14:35,648 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-69) task id 3b93fb1e-cd32-47f0-95e9-22fc4512675d is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:14:45,484 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-82) Setting new tasks map. The map contains now 4 tasks >2014-08-11 15:14:45,649 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-90) task id f7992afe-e86e-45cc-8d76-3177d5daead7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:14:45,650 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-90) task id 3b93fb1e-cd32-47f0-95e9-22fc4512675d is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:14:55,651 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-20) task id f7992afe-e86e-45cc-8d76-3177d5daead7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:14:55,651 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-20) task id 3b93fb1e-cd32-47f0-95e9-22fc4512675d is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:05,652 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) task id f7992afe-e86e-45cc-8d76-3177d5daead7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:05,653 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) task id 3b93fb1e-cd32-47f0-95e9-22fc4512675d has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:05,653 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) task id f7992afe-e86e-45cc-8d76-3177d5daead7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:05,654 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) task id 3b93fb1e-cd32-47f0-95e9-22fc4512675d has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:05,654 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-54) Polling and updating Async Tasks: 4 tasks, 1 tasks to poll now >2014-08-11 15:15:05,654 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) task id f7992afe-e86e-45cc-8d76-3177d5daead7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:05,655 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) task id 3b93fb1e-cd32-47f0-95e9-22fc4512675d has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:05,674 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) task id f7992afe-e86e-45cc-8d76-3177d5daead7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:05,674 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) task id 3b93fb1e-cd32-47f0-95e9-22fc4512675d has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:05,675 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) SPMAsyncTask::PollTask: Polling task 3b93fb1e-cd32-47f0-95e9-22fc4512675d (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:15:05,723 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) BaseAsyncTask::onTaskEndSuccess: Task 3b93fb1e-cd32-47f0-95e9-22fc4512675d (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:15:05,724 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-54) Task with DB Task ID e1fdbff2-4e1b-4d70-a800-04aaf2b741d4 and VDSM Task ID f7992afe-e86e-45cc-8d76-3177d5daead7 is in state Polling. End action for command bc76d541-1202-4e12-9294-0e93b0b6598a will proceed when all the entitys tasks are completed. >2014-08-11 15:15:05,724 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) task id f7992afe-e86e-45cc-8d76-3177d5daead7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:05,725 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) task id 3b93fb1e-cd32-47f0-95e9-22fc4512675d has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:05,726 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-54) Finished polling Tasks, will poll again in 10 seconds. >2014-08-11 15:15:15,485 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-83) Setting new tasks map. The map contains now 2 tasks >2014-08-11 15:15:15,727 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-75) task id f7992afe-e86e-45cc-8d76-3177d5daead7 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:15,728 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-75) task id f7992afe-e86e-45cc-8d76-3177d5daead7 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:15,729 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-75) task id 3b93fb1e-cd32-47f0-95e9-22fc4512675d has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:15,730 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-75) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 15:15:15,731 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-75) task id f7992afe-e86e-45cc-8d76-3177d5daead7 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:15,732 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-75) task id 3b93fb1e-cd32-47f0-95e9-22fc4512675d has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:15,740 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-75) task id f7992afe-e86e-45cc-8d76-3177d5daead7 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 15:15:15,741 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-75) SPMAsyncTask::PollTask: Polling task f7992afe-e86e-45cc-8d76-3177d5daead7 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:15:15,794 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-75) BaseAsyncTask::onTaskEndSuccess: Task f7992afe-e86e-45cc-8d76-3177d5daead7 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:15:15,794 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-75) CommandAsyncTask::endActionIfNecessary: All tasks of command bc76d541-1202-4e12-9294-0e93b0b6598a has ended -> executing endAction >2014-08-11 15:15:15,795 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-75) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: bc76d541-1202-4e12-9294-0e93b0b6598a): calling endAction . >2014-08-11 15:15:15,796 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-35) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RunVm, executionIndex: 0 >2014-08-11 15:15:15,810 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-35) Ending command successfully: org.ovirt.engine.core.bll.RunVmCommand >2014-08-11 15:15:15,833 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-35) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm completed, handling the result. >2014-08-11 15:15:15,833 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-35) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm succeeded, clearing tasks. >2014-08-11 15:15:15,843 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-35) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 3b93fb1e-cd32-47f0-95e9-22fc4512675d >2014-08-11 15:15:15,844 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 3b93fb1e-cd32-47f0-95e9-22fc4512675d), log id: 395a31c8 >2014-08-11 15:15:15,846 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=3b93fb1e-cd32-47f0-95e9-22fc4512675d), log id: 507a5ee2 >2014-08-11 15:15:15,963 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) FINISH, HSMClearTaskVDSCommand, log id: 507a5ee2 >2014-08-11 15:15:15,964 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) FINISH, SPMClearTaskVDSCommand, log id: 395a31c8 >2014-08-11 15:15:15,976 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-35) BaseAsyncTask::removeTaskFromDB: Removed task 3b93fb1e-cd32-47f0-95e9-22fc4512675d from DataBase >2014-08-11 15:15:15,976 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-35) SPMAsyncTask::ClearAsyncTask: Attempting to clear task f7992afe-e86e-45cc-8d76-3177d5daead7 >2014-08-11 15:15:15,977 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = f7992afe-e86e-45cc-8d76-3177d5daead7), log id: 3f2aff2 >2014-08-11 15:15:15,978 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=f7992afe-e86e-45cc-8d76-3177d5daead7), log id: 31fd3ae4 >2014-08-11 15:15:16,037 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) FINISH, HSMClearTaskVDSCommand, log id: 31fd3ae4 >2014-08-11 15:15:16,038 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-35) FINISH, SPMClearTaskVDSCommand, log id: 3f2aff2 >2014-08-11 15:15:16,084 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-35) BaseAsyncTask::removeTaskFromDB: Removed task f7992afe-e86e-45cc-8d76-3177d5daead7 from DataBase >2014-08-11 15:15:16,084 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-35) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity bc76d541-1202-4e12-9294-0e93b0b6598a >2014-08-11 15:15:41,072 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [206d1551] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 15:16:04,498 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-3) [2a5e4c4d] Failed to Acquire Lock to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 15:16:04,508 WARN [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-3) [2a5e4c4d] CanDoAction of action HibernateVm failed. Reasons:VAR__TYPE__VM,VAR__ACTION__HIBERNATE,ACTION_TYPE_FAILED_OBJECT_LOCKED >2014-08-11 15:16:05,215 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [206d1551] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:258) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:739) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 15:16:07,730 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [206d1551] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 15:28:36,930 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-11 15:28:37,017 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-11 15:28:37,018 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-11 15:28:37,020 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-11 15:28:37,021 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-11 15:28:37,022 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-11 15:28:37,024 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-11 15:28:37,025 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-11 15:28:37,025 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-11 15:28:37,026 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-11 15:28:37,027 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-11 15:28:37,027 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-11 15:28:37,028 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-11 15:28:37,028 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-11 15:28:37,029 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-11 15:28:37,029 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-11 15:28:37,030 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-11 15:28:37,030 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-11 15:28:37,031 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-11 15:28:37,032 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-11 15:28:37,032 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-11 15:28:37,033 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-11 15:28:37,033 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_USER" is "engine". >2014-08-11 15:28:37,034 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-11 15:28:37,035 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-11 15:28:37,036 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-11 15:28:37,036 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-11 15:28:37,037 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-11 15:28:37,037 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-11 15:28:37,038 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-11 15:28:37,041 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-11 15:28:37,041 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-11 15:28:37,042 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-11 15:28:37,042 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-11 15:28:37,044 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-11 15:28:37,045 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-11 15:28:37,045 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-11 15:28:37,046 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-11 15:28:37,046 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-11 15:28:37,047 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-11 15:28:37,048 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-11 15:28:37,048 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-11 15:28:37,049 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-11 15:28:37,050 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-11 15:28:37,051 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-11 15:28:37,052 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-11 15:28:37,054 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-11 15:28:37,056 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-11 15:28:37,056 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-11 15:28:37,057 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-11 15:28:37,057 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-11 15:28:37,058 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-11 15:28:37,058 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-11 15:28:37,058 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-11 15:28:37,059 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-11 15:28:37,059 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-11 15:28:37,060 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-11 15:28:37,060 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-11 15:28:37,060 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-11 15:28:37,061 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-11 15:28:37,061 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-11 15:28:37,062 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-11 15:28:37,062 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-11 15:28:37,062 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_USER" is "ovirt". >2014-08-11 15:28:37,063 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-11 15:28:37,063 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-11 15:28:37,063 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-11 15:28:37,064 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-11 15:28:37,065 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-11 15:28:37,067 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-11 15:28:37,601 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Start initializing Backend >2014-08-11 15:28:37,759 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 15:28:37,854 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 15:28:37,893 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-11 15:28:37,903 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-1) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-11 15:28:38,142 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Running ovirt-engine 3.5.0.0 >2014-08-11 15:28:38,143 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Start initializing dictionaries >2014-08-11 15:28:38,146 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Finished initializing dictionaries >2014-08-11 15:28:38,147 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing AuditLogCleanupManager >2014-08-11 15:28:38,148 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-11 15:28:38,165 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing AuditLogCleanupManager >2014-08-11 15:28:38,165 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing CommandEntityCleanupManager >2014-08-11 15:28:38,166 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-11 15:28:38,167 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing CommandEntityCleanupManager >2014-08-11 15:28:38,170 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Start initializing TagsDirector >2014-08-11 15:28:38,179 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Tag root added to tree >2014-08-11 15:28:38,246 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Finished initializing TagsDirector >2014-08-11 15:28:38,246 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Start initializing IsoDomainListSyncronizer >2014-08-11 15:28:38,254 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Finished initializing IsoDomainListSyncronizer >2014-08-11 15:28:38,342 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Completed initializing handlers >2014-08-11 15:28:38,363 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-08-11 15:28:38,373 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-11 15:28:38,374 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-08-11 15:28:38,374 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-08-11 15:28:38,375 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-08-11 15:28:38,375 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Mark incomplete jobs as UNKNOWN >2014-08-11 15:28:38,399 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Start initializing JobRepositoryCleanupManager >2014-08-11 15:28:38,399 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Finished initializing JobRepositoryCleanupManager >2014-08-11 15:28:38,400 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Start initializing AutoRecoveryManager >2014-08-11 15:28:38,401 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Finished initializing AutoRecoveryManager >2014-08-11 15:28:38,403 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Start initializing ExecutionMessageDirector >2014-08-11 15:28:38,404 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Finished initializing ExecutionMessageDirector >2014-08-11 15:28:38,456 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Start of enabled extensions list >2014-08-11 15:28:38,457 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-11 15:28:38,457 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-11 15:28:38,458 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) End of enabled extensions list >2014-08-11 15:28:38,554 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-2) Start initializing DbUserCacheManager >2014-08-11 15:28:38,555 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-2) Finished initializing DbUserCacheManager >2014-08-11 15:28:38,566 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-2) Initialization of AsyncTaskManager completed successfully. >2014-08-11 15:28:38,568 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-2) Start initializing ResourceManager >2014-08-11 15:28:38,590 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-11 15:28:38,643 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-2) Entered VdsManager constructor >2014-08-11 15:28:38,663 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-2) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-11 15:28:38,742 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-2) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-11 15:28:38,742 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-2) Entered VdsManager constructor >2014-08-11 15:28:38,754 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-2) Initialize vdsBroker (galactica,54,321) >2014-08-11 15:28:38,771 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-2) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-11 15:28:38,789 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-2) Finished initializing ResourceManager >2014-08-11 15:28:38,793 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-2) Initialization of OvfDataUpdater completed successfully. >2014-08-11 15:28:38,794 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Start scheduling to enable vds load balancer >2014-08-11 15:28:38,795 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Finished scheduling to enable vds load balancer >2014-08-11 15:28:38,795 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Start HA Reservation check >2014-08-11 15:28:38,796 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Finished HA Reservation check >2014-08-11 15:28:38,803 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-2) Init VM custom properties utilities >2014-08-11 15:28:38,804 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-2) Init device custom properties utilities >2014-08-11 15:28:38,810 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Initializing Scheduling manager >2014-08-11 15:28:38,829 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-11 15:28:38,830 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-11 15:28:38,830 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-11 15:28:38,844 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) External scheduler disabled, discovery skipped >2014-08-11 15:28:38,845 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Initialized Scheduling manager >2014-08-11 15:28:38,845 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-2) Initializing DWH Heart Beat >2014-08-11 15:28:38,846 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-2) DWH Heart Beat initialized >2014-08-11 15:28:41,787 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: h2 >2014-08-11 15:28:42,299 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 59eda1d7 >2014-08-11 15:28:42,308 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 59eda1d7 >2014-08-11 15:28:42,313 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-11 15:28:42,317 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: UNASSIGNED not exist in string table >2014-08-11 15:28:42,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-11 15:28:42,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-11 15:28:42,319 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-11 15:28:42,319 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-11 15:28:42,320 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-11 15:28:42,320 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-11 15:28:42,320 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-11 15:28:42,321 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-11 15:28:42,322 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-11 15:28:42,323 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-11 15:28:42,323 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STOPPED not exist in string table >2014-08-11 15:28:42,324 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STARTED not exist in string table >2014-08-11 15:28:42,324 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_ERROR not exist in string table >2014-08-11 15:28:42,325 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-11 15:28:42,325 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-11 15:28:42,326 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-11 15:28:42,326 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-11 15:28:42,498 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-11 15:28:42,499 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is running with disabled SELinux. >2014-08-11 15:28:42,826 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h1 >2014-08-11 15:28:42,978 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [13b5bee0] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-11 15:28:48,837 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-23) [6c840493] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-11 15:28:48,847 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [6c840493] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-11 15:28:48,854 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [6c840493] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-11 15:28:48,917 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [6c840493] Correlation ID: 6c840493, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-11 15:28:48,979 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) [6c840493] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-11 15:28:48,992 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) [6c840493] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-11 15:28:49,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [6c840493] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-11 15:28:49,028 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [6c840493] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 84963ea >2014-08-11 15:28:49,116 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [6c840493] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 52 GB of free space >2014-08-11 15:28:49,149 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [6c840493] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 52 GB of free space >2014-08-11 15:28:49,149 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [6c840493] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-11 15:28:49,155 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [6c840493] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 2049e713 >2014-08-11 15:28:49,162 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [6c840493] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 2049e713 >2014-08-11 15:28:49,162 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [6c840493] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 84963ea >2014-08-11 15:28:49,163 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [6c840493] Discovered no tasks on Storage Pool Default >2014-08-11 15:40:17,178 ERROR [org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter] (http--0.0.0.0-8080-1) Error in obtaining profile null >2014-08-11 15:40:41,825 ERROR [org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter] (http--0.0.0.0-8080-1) Failure in authentication to profile org.ovirt.engine.core.aaa.AuthenticationProfile@71a0e244. Invocation Result code is 0. Authn result code is 12 >2014-08-11 15:40:43,076 ERROR [org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter] (http--0.0.0.0-8080-1) Failure in authentication to profile org.ovirt.engine.core.aaa.AuthenticationProfile@71a0e244. Invocation Result code is 0. Authn result code is 12 >2014-08-11 15:40:46,547 ERROR [org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter] (http--0.0.0.0-8080-1) Failure in authentication to profile org.ovirt.engine.core.aaa.AuthenticationProfile@71a0e244. Invocation Result code is 0. Authn result code is 12 >2014-08-11 15:40:49,374 ERROR [org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter] (http--0.0.0.0-8080-1) Failure in authentication to profile org.ovirt.engine.core.aaa.AuthenticationProfile@71a0e244. Invocation Result code is 0. Authn result code is 12 >2014-08-11 15:42:43,220 ERROR [org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter] (http--0.0.0.0-8080-1) Failure in authentication to profile org.ovirt.engine.core.aaa.AuthenticationProfile@71a0e244. Invocation Result code is 0. Authn result code is 12 >2014-08-11 15:43:24,192 ERROR [org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter] (http--0.0.0.0-8080-1) Failure in authentication to profile org.ovirt.engine.core.aaa.AuthenticationProfile@71a0e244. Invocation Result code is 0. Authn result code is 12 >2014-08-11 15:43:53,824 ERROR [org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter] (http--0.0.0.0-8080-1) Error in obtaining profile null >2014-08-11 15:43:58,914 ERROR [org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter] (http--0.0.0.0-8080-1) Error in obtaining profile null >2014-08-11 15:45:23,192 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-1) Running command: LoginUserCommand internal: false. >2014-08-11 15:45:23,208 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-11 15:45:23,341 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-1) [1cf4f4b4] Running command: LogoutBySessionCommand internal: false. >2014-08-11 15:45:23,348 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-1) [149e6341] Running command: LogoutUserCommand internal: false. >2014-08-11 15:45:23,366 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [149e6341] Correlation ID: 149e6341, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-11 15:46:09,573 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-4) Running command: LoginAdminUserCommand internal: false. >2014-08-11 15:46:09,590 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-11 15:46:12,466 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-2) Running command: LoginUserCommand internal: false. >2014-08-11 15:47:59,194 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-4) Running command: LoginUserCommand internal: false. >2014-08-11 15:47:59,224 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-11 15:47:59,250 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-4) [195792b9] Running command: LogoutBySessionCommand internal: false. >2014-08-11 15:47:59,253 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-4) [2fbd43c9] Running command: LogoutUserCommand internal: false. >2014-08-11 15:47:59,265 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [2fbd43c9] Correlation ID: 2fbd43c9, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-11 15:49:56,734 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-4) Running command: LoginUserCommand internal: false. >2014-08-11 15:49:56,740 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-11 15:49:56,759 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-4) [7adf3309] Running command: LogoutBySessionCommand internal: false. >2014-08-11 15:49:56,762 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-4) [3596d9d3] Running command: LogoutUserCommand internal: false. >2014-08-11 15:49:56,774 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [3596d9d3] Correlation ID: 3596d9d3, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-11 15:50:43,998 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-4) Running command: LoginUserCommand internal: false. >2014-08-11 15:50:44,004 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-11 15:50:44,057 INFO [org.ovirt.engine.core.bll.MoveDisksCommand] (http--0.0.0.0-8080-4) [62348920] Running command: MoveDisksCommand internal: false. Entities affected : ID: b5576e45-7f13-465f-b26f-05d52faf28d1 Type: DiskAction group CONFIGURE_DISK_STORAGE with role type USER >2014-08-11 15:50:44,140 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (http--0.0.0.0-8080-4) [62348920] Lock Acquired to object EngineLock [exclusiveLocks= key: b5576e45-7f13-465f-b26f-05d52faf28d1 value: DISK >, sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-11 15:50:44,193 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (org.ovirt.thread.pool-8-thread-5) [62348920] Running command: LiveMigrateVmDisksCommand Task handler: LiveSnapshotTaskHandler internal: false. Entities affected : ID: b5576e45-7f13-465f-b26f-05d52faf28d1 Type: DiskAction group CONFIGURE_DISK_STORAGE with role type USER, ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 15:50:44,194 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-4) [8c2ef7a] Running command: LogoutBySessionCommand internal: false. >2014-08-11 15:50:44,197 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-4) [45e98b61] Running command: LogoutUserCommand internal: false. >2014-08-11 15:50:44,211 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [45e98b61] Correlation ID: 45e98b61, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-11 15:50:44,279 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-5) Command b27a53b5-f879-49b6-8db8-a62a550031b6 persisting async task placeholder for child command 6b3def1d-349e-4e2d-96bc-944c05979b30 >2014-08-11 15:50:44,451 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-5) Running command: CreateAllSnapshotsFromVmCommand internal: true. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER >2014-08-11 15:50:44,617 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-5) [442fe14f] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-08-11 15:50:44,625 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-5) [442fe14f] START, CreateSnapshotVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageSizeInBytes = 10737418240, volumeFormat = COW, newImageId = 3812059c-f566-4975-955f-9937a0885b61, newImageDescription = , imageId = d7c1ba6b-4a30-415d-ba40-1d5499786ae8, sourceImageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1), log id: 496a6290 >2014-08-11 15:50:44,631 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-5) [442fe14f] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 15:50:44,976 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-5) [442fe14f] FINISH, CreateSnapshotVDSCommand, return: 3812059c-f566-4975-955f-9937a0885b61, log id: 496a6290 >2014-08-11 15:50:45,028 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-5) [442fe14f] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 0d114fed-566c-4559-bbdf-dafbdabd21c2 >2014-08-11 15:50:45,029 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-5) [442fe14f] CommandMultiAsyncTasks::AttachTask: Attaching task d176df03-f48b-4dde-884a-f04440f49e81 to command 0d114fed-566c-4559-bbdf-dafbdabd21c2. >2014-08-11 15:50:45,056 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-5) [442fe14f] Adding task d176df03-f48b-4dde-884a-f04440f49e81 (Parent Command LiveMigrateVmDisks, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:50:45,156 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-5) Correlation ID: 7b7835b9, Job ID: a5216d41-ef35-4dbf-ab5b-4e13012a9c03, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'Auto-generated for Live Storage Migration' creation for VM 'vm3' was initiated by admin. >2014-08-11 15:50:45,164 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-5) BaseAsyncTask::startPollingTask: Starting to poll task d176df03-f48b-4dde-884a-f04440f49e81. >2014-08-11 15:50:48,574 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-100) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-11 15:50:48,587 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-100) SPMAsyncTask::PollTask: Polling task d176df03-f48b-4dde-884a-f04440f49e81 (Parent Command LiveMigrateVmDisks, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:50:48,668 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-100) BaseAsyncTask::onTaskEndSuccess: Task d176df03-f48b-4dde-884a-f04440f49e81 (Parent Command LiveMigrateVmDisks, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:50:48,669 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-100) CommandAsyncTask::endActionIfNecessary: All tasks of command 0d114fed-566c-4559-bbdf-dafbdabd21c2 has ended -> executing endAction >2014-08-11 15:50:48,670 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-100) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 0d114fed-566c-4559-bbdf-dafbdabd21c2): calling endAction . >2014-08-11 15:50:48,671 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-8) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction LiveMigrateVmDisks, executionIndex: 0 >2014-08-11 15:50:48,690 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (org.ovirt.thread.pool-8-thread-8) Ending command successfully: org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand >2014-08-11 15:50:48,711 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-8) Ending command successfully: org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand >2014-08-11 15:50:48,753 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-8) START, SnapshotVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327), log id: 68c6a5d2 >2014-08-11 15:50:58,837 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-8) FINISH, SnapshotVDSCommand, log id: 68c6a5d2 >2014-08-11 15:50:58,872 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-8) [442fe14f] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotCommand >2014-08-11 15:50:58,878 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-8) [442fe14f] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageId = 3812059c-f566-4975-955f-9937a0885b61), log id: 7504774a >2014-08-11 15:50:58,910 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-8) [442fe14f] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@5bf84b4, log id: 7504774a >2014-08-11 15:50:58,946 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'Auto-generated for Live Storage Migration' creation for VM 'vm3' has been completed. >2014-08-11 15:50:58,965 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (org.ovirt.thread.pool-8-thread-8) Running command: LiveMigrateVmDisksCommand Task handler: LiveMigrateDisksTaskHandler internal: false. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 15:50:59,059 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-8) [6c687112] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-11 15:50:59,079 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-8) [6c687112] Running command: LiveMigrateDiskCommand Task handler: CreateImagePlaceholderTaskHandler internal: true. Entities affected : ID: b5576e45-7f13-465f-b26f-05d52faf28d1 Type: DiskAction group CONFIGURE_DISK_STORAGE with role type USER, ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 15:50:59,105 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CloneImageGroupStructureVDSCommand] (org.ovirt.thread.pool-8-thread-8) [6c687112] START, CloneImageGroupStructureVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, dstDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a), log id: b8c8fcc >2014-08-11 15:50:59,521 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CloneImageGroupStructureVDSCommand] (org.ovirt.thread.pool-8-thread-8) [6c687112] FINISH, CloneImageGroupStructureVDSCommand, log id: b8c8fcc >2014-08-11 15:50:59,652 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-8) [6c687112] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 0526b86b-6cd9-4820-bc3e-4bfe48d481b7 >2014-08-11 15:50:59,653 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-8) [6c687112] CommandMultiAsyncTasks::AttachTask: Attaching task 96e8353c-4942-4e91-8e34-a9486eded603 to command 0526b86b-6cd9-4820-bc3e-4bfe48d481b7. >2014-08-11 15:50:59,692 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-8) [6c687112] Adding task 96e8353c-4942-4e91-8e34-a9486eded603 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:50:59,733 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-8) [6c687112] Correlation ID: 6c687112, Job ID: a075caca-326f-4214-b402-b3f886178832, Call Stack: null, Custom Event ID: -1, Message: User admin moving disk vm3_Disk1 to domain sd_35. >2014-08-11 15:50:59,734 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-8) [6c687112] BaseAsyncTask::startPollingTask: Starting to poll task 96e8353c-4942-4e91-8e34-a9486eded603. >2014-08-11 15:50:59,736 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-8) [6c687112] BaseAsyncTask::startPollingTask: Starting to poll task 96e8353c-4942-4e91-8e34-a9486eded603. >2014-08-11 15:50:59,742 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (org.ovirt.thread.pool-8-thread-8) [6c687112] Lock freed to object EngineLock [exclusiveLocks= key: b5576e45-7f13-465f-b26f-05d52faf28d1 value: DISK >, sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-11 15:50:59,742 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-8) [6c687112] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateVmDisks completed, handling the result. >2014-08-11 15:50:59,743 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-8) [6c687112] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateVmDisks succeeded, clearing tasks. >2014-08-11 15:50:59,750 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-8) [6c687112] SPMAsyncTask::ClearAsyncTask: Attempting to clear task d176df03-f48b-4dde-884a-f04440f49e81 >2014-08-11 15:50:59,751 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-8) [6c687112] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = d176df03-f48b-4dde-884a-f04440f49e81), log id: 65da066a >2014-08-11 15:50:59,785 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-8) [6c687112] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=d176df03-f48b-4dde-884a-f04440f49e81), log id: 2117e279 >2014-08-11 15:50:59,892 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-8) [6c687112] FINISH, HSMClearTaskVDSCommand, log id: 2117e279 >2014-08-11 15:50:59,893 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-8) [6c687112] FINISH, SPMClearTaskVDSCommand, log id: 65da066a >2014-08-11 15:50:59,916 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-8) [6c687112] BaseAsyncTask::removeTaskFromDB: Removed task d176df03-f48b-4dde-884a-f04440f49e81 from DataBase >2014-08-11 15:50:59,917 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-8) [6c687112] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 0d114fed-566c-4559-bbdf-dafbdabd21c2 >2014-08-11 15:51:07,415 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-52) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: 78ce00c8 >2014-08-11 15:51:07,425 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-52) FINISH, FullListVdsCommand, return: [{displaySecurePort=5900, kvmEnable=true, nicModel=rtl8139,pv, guestIPs=, keyboardLayout=en-us, displayIp=10.35.0.105, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, username=Unknown, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, status=Up, emulatedMachine=pc-1.0, vmType=kvm, memGuaranteedSize=1024, display=qxl, pid=25652, smartcardEnable=false, bootMenuEnable=false, numaTune={mode=interleave}, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, guestFQDN=, smpCoresPerSocket=1, maxVCpus=160, clientIp=, devices=[Ljava.lang.Object;@1c95886e, vmName=vm3, fileTransferEnable=true, cpuType=SandyBridge}], log id: 78ce00c8 >2014-08-11 15:51:07,429 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Received a memballoon Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 15:51:07,430 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Received a spice Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5900, type=graphics} >2014-08-11 15:51:08,672 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-55) Polling and updating Async Tasks: 2 tasks, 1 tasks to poll now >2014-08-11 15:51:08,680 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) SPMAsyncTask::PollTask: Polling task 96e8353c-4942-4e91-8e34-a9486eded603 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:51:08,763 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) BaseAsyncTask::onTaskEndSuccess: Task 96e8353c-4942-4e91-8e34-a9486eded603 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:51:08,765 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-55) CommandAsyncTask::endActionIfNecessary: All tasks of command 0526b86b-6cd9-4820-bc3e-4bfe48d481b7 has ended -> executing endAction >2014-08-11 15:51:08,766 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-55) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 0526b86b-6cd9-4820-bc3e-4bfe48d481b7): calling endAction . >2014-08-11 15:51:08,767 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction LiveMigrateDisk, executionIndex: 0 >2014-08-11 15:51:08,885 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] Ending command successfully: org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand >2014-08-11 15:51:08,902 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] Running command: LiveMigrateDiskCommand Task handler: VmReplicateDiskStartTaskHandler internal: false. Entities affected : ID: b5576e45-7f13-465f-b26f-05d52faf28d1 Type: DiskAction group CONFIGURE_DISK_STORAGE with role type USER, ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 15:51:08,905 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] START, VmReplicateDiskStartVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327), log id: 45941308 >2014-08-11 15:51:09,004 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] FINISH, VmReplicateDiskStartVDSCommand, log id: 45941308 >2014-08-11 15:51:09,035 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SyncImageGroupDataVDSCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] START, SyncImageGroupDataVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, dstDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, syncType=INTERNAL), log id: 74883ca8 >2014-08-11 15:51:09,413 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SyncImageGroupDataVDSCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] FINISH, SyncImageGroupDataVDSCommand, log id: 74883ca8 >2014-08-11 15:51:09,426 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-34) [6c687112] CommandMultiAsyncTasks::AttachTask: Attaching task 2cf6c526-f9b2-4bbb-9eaf-0b46f9e47739 to command 0526b86b-6cd9-4820-bc3e-4bfe48d481b7. >2014-08-11 15:51:09,448 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-34) [6c687112] Adding task 2cf6c526-f9b2-4bbb-9eaf-0b46f9e47739 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:51:09,473 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) [6c687112] BaseAsyncTask::startPollingTask: Starting to poll task 2cf6c526-f9b2-4bbb-9eaf-0b46f9e47739. >2014-08-11 15:51:09,474 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) [6c687112] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateDisk completed, handling the result. >2014-08-11 15:51:09,474 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) [6c687112] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateDisk succeeded, clearing tasks. >2014-08-11 15:51:09,475 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) [6c687112] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 96e8353c-4942-4e91-8e34-a9486eded603 >2014-08-11 15:51:09,476 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 96e8353c-4942-4e91-8e34-a9486eded603), log id: b4d002b >2014-08-11 15:51:09,478 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=96e8353c-4942-4e91-8e34-a9486eded603), log id: 6af57070 >2014-08-11 15:51:09,642 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] FINISH, HSMClearTaskVDSCommand, log id: 6af57070 >2014-08-11 15:51:09,643 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] FINISH, SPMClearTaskVDSCommand, log id: b4d002b >2014-08-11 15:51:09,655 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) [6c687112] BaseAsyncTask::removeTaskFromDB: Removed task 96e8353c-4942-4e91-8e34-a9486eded603 from DataBase >2014-08-11 15:51:09,656 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-34) [6c687112] [within thread]: Some of the tasks related to command id 0526b86b-6cd9-4820-bc3e-4bfe48d481b7 were not cleared yet (Task id 2cf6c526-f9b2-4bbb-9eaf-0b46f9e47739 is in state Polling). >2014-08-11 15:51:18,768 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-86) Polling and updating Async Tasks: 3 tasks, 1 tasks to poll now >2014-08-11 15:51:18,777 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-86) SPMAsyncTask::PollTask: Polling task 2cf6c526-f9b2-4bbb-9eaf-0b46f9e47739 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-11 15:51:18,779 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-86) Finished polling Tasks, will poll again in 10 seconds. >2014-08-11 15:52:08,575 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-19) Setting new tasks map. The map contains now 2 tasks >2014-08-11 15:52:08,808 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-23) [6c840493] Polling and updating Async Tasks: 2 tasks, 1 tasks to poll now >2014-08-11 15:52:08,841 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-23) [6c840493] Finished polling Tasks, will poll again in 10 seconds. >2014-08-11 15:52:38,577 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-2) Setting new tasks map. The map contains now 1 tasks >2014-08-11 15:52:40,768 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-20) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-11 15:52:41,084 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-4) Running command: LoginUserCommand internal: false. >2014-08-11 15:52:41,090 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-11 15:52:41,105 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-4) [7dac9aa5] Running command: LogoutBySessionCommand internal: false. >2014-08-11 15:52:41,107 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-4) [5b9b1539] Running command: LogoutUserCommand internal: false. >2014-08-11 15:52:41,115 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [5b9b1539] Correlation ID: 5b9b1539, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-11 15:52:42,215 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-20) Finished polling Tasks, will poll again in 10 seconds. >2014-08-11 15:53:03,942 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-73) SPMAsyncTask::PollTask: Polling task 2cf6c526-f9b2-4bbb-9eaf-0b46f9e47739 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:53:03,959 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-73) BaseAsyncTask::onTaskEndSuccess: Task 2cf6c526-f9b2-4bbb-9eaf-0b46f9e47739 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:53:03,961 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-73) CommandAsyncTask::endActionIfNecessary: All tasks of command 0526b86b-6cd9-4820-bc3e-4bfe48d481b7 has ended -> executing endAction >2014-08-11 15:53:03,961 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-73) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 0526b86b-6cd9-4820-bc3e-4bfe48d481b7): calling endAction . >2014-08-11 15:53:03,962 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-20) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction LiveMigrateDisk, executionIndex: 1 >2014-08-11 15:53:04,041 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] Ending command successfully: org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand >2014-08-11 15:53:04,042 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] Running command: LiveMigrateDiskCommand Task handler: VmReplicateDiskFinishTaskHandler internal: false. Entities affected : ID: b5576e45-7f13-465f-b26f-05d52faf28d1 Type: DiskAction group CONFIGURE_DISK_STORAGE with role type USER, ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 15:53:04,068 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] START, VmReplicateDiskFinishVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327), log id: 65d6b8cc >2014-08-11 15:53:04,233 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] FINISH, VmReplicateDiskFinishVDSCommand, log id: 65d6b8cc >2014-08-11 15:53:04,241 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageId = d7c1ba6b-4a30-415d-ba40-1d5499786ae8), log id: 707390d2 >2014-08-11 15:53:04,275 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@3d3cc48, log id: 707390d2 >2014-08-11 15:53:04,291 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageId = 5a8224b1-9aa1-4e55-b26f-2f72f91b8fcc), log id: 56420925 >2014-08-11 15:53:04,323 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@84de549e, log id: 56420925 >2014-08-11 15:53:04,332 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageId = 3812059c-f566-4975-955f-9937a0885b61), log id: 7892147c >2014-08-11 15:53:04,361 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@82964363, log id: 7892147c >2014-08-11 15:53:04,410 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, postZeros = false, forceDelete = false), log id: 40fd87a8 >2014-08-11 15:53:05,838 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] FINISH, DeleteImageGroupVDSCommand, log id: 40fd87a8 >2014-08-11 15:53:05,884 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-20) [6c687112] CommandMultiAsyncTasks::AttachTask: Attaching task d89ca733-1fbe-4236-955b-3ab924b9e945 to command 0526b86b-6cd9-4820-bc3e-4bfe48d481b7. >2014-08-11 15:53:05,906 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-20) [6c687112] Adding task d89ca733-1fbe-4236-955b-3ab924b9e945 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:53:05,931 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-20) [6c687112] BaseAsyncTask::startPollingTask: Starting to poll task d89ca733-1fbe-4236-955b-3ab924b9e945. >2014-08-11 15:53:05,931 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-20) [6c687112] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateDisk completed, handling the result. >2014-08-11 15:53:05,932 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-20) [6c687112] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateDisk succeeded, clearing tasks. >2014-08-11 15:53:05,933 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-20) [6c687112] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 2cf6c526-f9b2-4bbb-9eaf-0b46f9e47739 >2014-08-11 15:53:05,934 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 2cf6c526-f9b2-4bbb-9eaf-0b46f9e47739), log id: 1df758de >2014-08-11 15:53:05,935 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=2cf6c526-f9b2-4bbb-9eaf-0b46f9e47739), log id: 59eb6630 >2014-08-11 15:53:06,170 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] FINISH, HSMClearTaskVDSCommand, log id: 59eb6630 >2014-08-11 15:53:06,171 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-20) [6c687112] FINISH, SPMClearTaskVDSCommand, log id: 1df758de >2014-08-11 15:53:06,187 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-20) [6c687112] BaseAsyncTask::removeTaskFromDB: Removed task 2cf6c526-f9b2-4bbb-9eaf-0b46f9e47739 from DataBase >2014-08-11 15:53:06,188 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-20) [6c687112] [within thread]: Some of the tasks related to command id 0526b86b-6cd9-4820-bc3e-4bfe48d481b7 were not cleared yet (Task id d89ca733-1fbe-4236-955b-3ab924b9e945 is in state Polling). >2014-08-11 15:53:09,294 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-92) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: 1593a19e >2014-08-11 15:53:09,305 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-92) FINISH, FullListVdsCommand, return: [{displaySecurePort=5900, kvmEnable=true, nicModel=rtl8139,pv, guestIPs=, keyboardLayout=en-us, displayIp=10.35.0.105, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, username=Unknown, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, status=Up, emulatedMachine=pc-1.0, vmType=kvm, memGuaranteedSize=1024, display=qxl, pid=25652, smartcardEnable=false, bootMenuEnable=false, numaTune={mode=interleave}, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, guestFQDN=, smpCoresPerSocket=1, maxVCpus=160, clientIp=, devices=[Ljava.lang.Object;@3f8382b3, vmName=vm3, fileTransferEnable=true, cpuType=SandyBridge}], log id: 1593a19e >2014-08-11 15:53:09,310 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Received a memballoon Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 15:53:09,311 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Received a spice Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5900, type=graphics} >2014-08-11 15:53:13,962 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-3) Polling and updating Async Tasks: 2 tasks, 1 tasks to poll now >2014-08-11 15:53:14,021 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) SPMAsyncTask::PollTask: Polling task d89ca733-1fbe-4236-955b-3ab924b9e945 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:53:14,031 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) BaseAsyncTask::onTaskEndSuccess: Task d89ca733-1fbe-4236-955b-3ab924b9e945 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:53:14,032 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-3) CommandAsyncTask::endActionIfNecessary: All tasks of command 0526b86b-6cd9-4820-bc3e-4bfe48d481b7 has ended -> executing endAction >2014-08-11 15:53:14,033 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-3) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 0526b86b-6cd9-4820-bc3e-4bfe48d481b7): calling endAction . >2014-08-11 15:53:14,034 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction LiveMigrateDisk, executionIndex: 2 >2014-08-11 15:53:14,103 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] Ending command successfully: org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand >2014-08-11 15:53:14,104 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-11 15:53:14,155 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-34) [6c687112] Correlation ID: 6c687112, Call Stack: null, Custom Event ID: -1, Message: User admin finished moving disk vm3_Disk1 to domain sd_35. >2014-08-11 15:53:14,156 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) [6c687112] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateDisk completed, handling the result. >2014-08-11 15:53:14,156 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) [6c687112] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateDisk succeeded, clearing tasks. >2014-08-11 15:53:14,163 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) [6c687112] SPMAsyncTask::ClearAsyncTask: Attempting to clear task d89ca733-1fbe-4236-955b-3ab924b9e945 >2014-08-11 15:53:14,164 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = d89ca733-1fbe-4236-955b-3ab924b9e945), log id: 5d85de48 >2014-08-11 15:53:14,166 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=d89ca733-1fbe-4236-955b-3ab924b9e945), log id: 7fc478ef >2014-08-11 15:53:14,230 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] FINISH, HSMClearTaskVDSCommand, log id: 7fc478ef >2014-08-11 15:53:14,231 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [6c687112] FINISH, SPMClearTaskVDSCommand, log id: 5d85de48 >2014-08-11 15:53:14,246 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) [6c687112] BaseAsyncTask::removeTaskFromDB: Removed task d89ca733-1fbe-4236-955b-3ab924b9e945 from DataBase >2014-08-11 15:53:14,247 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) [6c687112] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 0526b86b-6cd9-4820-bc3e-4bfe48d481b7 >2014-08-11 15:54:08,580 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-59) Setting new tasks map. The map contains now 1 tasks >2014-08-11 15:54:20,867 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-4) Running command: LoginUserCommand internal: false. >2014-08-11 15:54:20,874 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-11 15:54:20,887 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-4) [4cee82c] Running command: LogoutBySessionCommand internal: false. >2014-08-11 15:54:20,889 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-4) [64723574] Running command: LogoutUserCommand internal: false. >2014-08-11 15:54:20,898 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [64723574] Correlation ID: 64723574, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-11 15:54:38,582 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-39) Setting new tasks map. The map contains now 0 tasks >2014-08-11 15:54:38,583 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-39) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 15:54:39,525 INFO [org.ovirt.engine.core.bll.MoveDisksCommand] (http--0.0.0.0-8080-4) [65f6bf77] Running command: MoveDisksCommand internal: false. Entities affected : ID: b5576e45-7f13-465f-b26f-05d52faf28d1 Type: DiskAction group CONFIGURE_DISK_STORAGE with role type USER >2014-08-11 15:54:39,584 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (http--0.0.0.0-8080-4) [65f6bf77] Lock Acquired to object EngineLock [exclusiveLocks= key: b5576e45-7f13-465f-b26f-05d52faf28d1 value: DISK >, sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-11 15:54:39,626 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (org.ovirt.thread.pool-8-thread-29) [65f6bf77] Running command: LiveMigrateVmDisksCommand Task handler: LiveSnapshotTaskHandler internal: false. Entities affected : ID: b5576e45-7f13-465f-b26f-05d52faf28d1 Type: DiskAction group CONFIGURE_DISK_STORAGE with role type USER, ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 15:54:39,731 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-29) Command 8967ca4c-0bf8-4f40-87b9-198af98ac344 persisting async task placeholder for child command 59b8da20-570a-4455-afb8-e9f7c87b2abf >2014-08-11 15:54:39,783 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-29) Running command: CreateAllSnapshotsFromVmCommand internal: true. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER >2014-08-11 15:54:39,833 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-29) [2443457b] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-08-11 15:54:39,837 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-29) [2443457b] START, CreateSnapshotVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageSizeInBytes = 10737418240, volumeFormat = COW, newImageId = 26308abd-8540-410e-8018-58cd2157084d, newImageDescription = , imageId = 3812059c-f566-4975-955f-9937a0885b61, sourceImageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1), log id: 3f08c7ed >2014-08-11 15:54:39,839 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-29) [2443457b] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 15:54:40,205 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-29) [2443457b] FINISH, CreateSnapshotVDSCommand, return: 26308abd-8540-410e-8018-58cd2157084d, log id: 3f08c7ed >2014-08-11 15:54:40,308 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-29) [2443457b] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command a50555b6-3974-4e67-8bbd-9abb1f96c758 >2014-08-11 15:54:40,308 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-29) [2443457b] CommandMultiAsyncTasks::AttachTask: Attaching task e7b5e918-f0f7-4123-b4a4-0ca147dc4752 to command a50555b6-3974-4e67-8bbd-9abb1f96c758. >2014-08-11 15:54:40,330 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-29) [2443457b] Adding task e7b5e918-f0f7-4123-b4a4-0ca147dc4752 (Parent Command LiveMigrateVmDisks, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:54:40,396 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-29) Correlation ID: 65ded68d, Job ID: c6f4a491-1f9c-4083-b526-fb651d8b8a81, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'Auto-generated for Live Storage Migration' creation for VM 'vm3' was initiated by admin. >2014-08-11 15:54:40,405 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-29) BaseAsyncTask::startPollingTask: Starting to poll task e7b5e918-f0f7-4123-b4a4-0ca147dc4752. >2014-08-11 15:54:44,037 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-56) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-11 15:54:44,043 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-56) SPMAsyncTask::PollTask: Polling task e7b5e918-f0f7-4123-b4a4-0ca147dc4752 (Parent Command LiveMigrateVmDisks, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:54:44,065 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-56) BaseAsyncTask::onTaskEndSuccess: Task e7b5e918-f0f7-4123-b4a4-0ca147dc4752 (Parent Command LiveMigrateVmDisks, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:54:44,066 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-56) CommandAsyncTask::endActionIfNecessary: All tasks of command a50555b6-3974-4e67-8bbd-9abb1f96c758 has ended -> executing endAction >2014-08-11 15:54:44,066 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-56) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: a50555b6-3974-4e67-8bbd-9abb1f96c758): calling endAction . >2014-08-11 15:54:44,067 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction LiveMigrateVmDisks, executionIndex: 0 >2014-08-11 15:54:44,082 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (org.ovirt.thread.pool-8-thread-49) Ending command successfully: org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand >2014-08-11 15:54:44,102 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-49) Ending command successfully: org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand >2014-08-11 15:54:44,114 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-49) START, SnapshotVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327), log id: 1336b658 >2014-08-11 15:54:53,843 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-49) FINISH, SnapshotVDSCommand, log id: 1336b658 >2014-08-11 15:54:53,880 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-49) [2443457b] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotCommand >2014-08-11 15:54:53,889 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-49) [2443457b] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageId = 26308abd-8540-410e-8018-58cd2157084d), log id: 42893e06 >2014-08-11 15:54:53,948 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-49) [2443457b] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@2c360664, log id: 42893e06 >2014-08-11 15:54:53,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'Auto-generated for Live Storage Migration' creation for VM 'vm3' has been completed. >2014-08-11 15:54:53,989 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (org.ovirt.thread.pool-8-thread-49) Running command: LiveMigrateVmDisksCommand Task handler: LiveMigrateDisksTaskHandler internal: false. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 15:54:54,070 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-49) [2d37732c] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-11 15:54:54,095 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-49) [2d37732c] Running command: LiveMigrateDiskCommand Task handler: CreateImagePlaceholderTaskHandler internal: true. Entities affected : ID: b5576e45-7f13-465f-b26f-05d52faf28d1 Type: DiskAction group CONFIGURE_DISK_STORAGE with role type USER, ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 15:54:54,112 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CloneImageGroupStructureVDSCommand] (org.ovirt.thread.pool-8-thread-49) [2d37732c] START, CloneImageGroupStructureVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, dstDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f), log id: 1fd462a6 >2014-08-11 15:54:54,576 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CloneImageGroupStructureVDSCommand] (org.ovirt.thread.pool-8-thread-49) [2d37732c] FINISH, CloneImageGroupStructureVDSCommand, log id: 1fd462a6 >2014-08-11 15:54:54,644 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) [2d37732c] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 3b7c02b8-97d6-4b4e-b584-056c4573183a >2014-08-11 15:54:54,645 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-49) [2d37732c] CommandMultiAsyncTasks::AttachTask: Attaching task 0e064b4b-978c-4428-9738-b602124c6587 to command 3b7c02b8-97d6-4b4e-b584-056c4573183a. >2014-08-11 15:54:54,667 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-49) [2d37732c] Adding task 0e064b4b-978c-4428-9738-b602124c6587 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:54:54,700 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-49) [2d37732c] Correlation ID: 2d37732c, Job ID: 60be616f-3af5-4b3e-9661-f4e1c4699195, Call Stack: null, Custom Event ID: -1, Message: User admin moving disk vm3_Disk1 to domain sd35_1. >2014-08-11 15:54:54,701 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-49) [2d37732c] BaseAsyncTask::startPollingTask: Starting to poll task 0e064b4b-978c-4428-9738-b602124c6587. >2014-08-11 15:54:54,706 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-49) [2d37732c] BaseAsyncTask::startPollingTask: Starting to poll task 0e064b4b-978c-4428-9738-b602124c6587. >2014-08-11 15:54:54,717 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (org.ovirt.thread.pool-8-thread-49) [2d37732c] Lock freed to object EngineLock [exclusiveLocks= key: b5576e45-7f13-465f-b26f-05d52faf28d1 value: DISK >, sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-11 15:54:54,718 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) [2d37732c] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateVmDisks completed, handling the result. >2014-08-11 15:54:54,718 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) [2d37732c] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateVmDisks succeeded, clearing tasks. >2014-08-11 15:54:54,725 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-49) [2d37732c] SPMAsyncTask::ClearAsyncTask: Attempting to clear task e7b5e918-f0f7-4123-b4a4-0ca147dc4752 >2014-08-11 15:54:54,725 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) [2d37732c] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = e7b5e918-f0f7-4123-b4a4-0ca147dc4752), log id: 4e904cb8 >2014-08-11 15:54:54,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) [2d37732c] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=e7b5e918-f0f7-4123-b4a4-0ca147dc4752), log id: 639cec53 >2014-08-11 15:54:54,804 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) [2d37732c] FINISH, HSMClearTaskVDSCommand, log id: 639cec53 >2014-08-11 15:54:54,805 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) [2d37732c] FINISH, SPMClearTaskVDSCommand, log id: 4e904cb8 >2014-08-11 15:54:54,816 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-49) [2d37732c] BaseAsyncTask::removeTaskFromDB: Removed task e7b5e918-f0f7-4123-b4a4-0ca147dc4752 from DataBase >2014-08-11 15:54:54,817 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) [2d37732c] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity a50555b6-3974-4e67-8bbd-9abb1f96c758 >2014-08-11 15:54:56,259 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-94) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: 54549e4a >2014-08-11 15:54:56,276 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-94) FINISH, FullListVdsCommand, return: [{displaySecurePort=5900, kvmEnable=true, nicModel=rtl8139,pv, guestIPs=, keyboardLayout=en-us, displayIp=10.35.0.105, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, username=Unknown, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, status=Up, emulatedMachine=pc-1.0, vmType=kvm, memGuaranteedSize=1024, display=qxl, pid=25652, smartcardEnable=false, bootMenuEnable=false, numaTune={mode=interleave}, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, guestFQDN=, smpCoresPerSocket=1, maxVCpus=160, clientIp=, devices=[Ljava.lang.Object;@7f643786, vmName=vm3, fileTransferEnable=true, cpuType=SandyBridge}], log id: 54549e4a >2014-08-11 15:54:56,279 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Received a memballoon Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 15:54:56,279 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Received a spice Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5900, type=graphics} >2014-08-11 15:55:04,068 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-10) Polling and updating Async Tasks: 2 tasks, 1 tasks to poll now >2014-08-11 15:55:04,117 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-10) SPMAsyncTask::PollTask: Polling task 0e064b4b-978c-4428-9738-b602124c6587 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:55:04,135 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-10) BaseAsyncTask::onTaskEndSuccess: Task 0e064b4b-978c-4428-9738-b602124c6587 (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:55:04,136 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-10) CommandAsyncTask::endActionIfNecessary: All tasks of command 3b7c02b8-97d6-4b4e-b584-056c4573183a has ended -> executing endAction >2014-08-11 15:55:04,136 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-10) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 3b7c02b8-97d6-4b4e-b584-056c4573183a): calling endAction . >2014-08-11 15:55:04,137 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-19) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction LiveMigrateDisk, executionIndex: 0 >2014-08-11 15:55:04,193 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-19) [2d37732c] Ending command successfully: org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand >2014-08-11 15:55:04,201 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-19) [2d37732c] Running command: LiveMigrateDiskCommand Task handler: VmReplicateDiskStartTaskHandler internal: false. Entities affected : ID: b5576e45-7f13-465f-b26f-05d52faf28d1 Type: DiskAction group CONFIGURE_DISK_STORAGE with role type USER, ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 15:55:04,204 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (org.ovirt.thread.pool-8-thread-19) [2d37732c] START, VmReplicateDiskStartVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327), log id: 3f14f8ef >2014-08-11 15:55:04,337 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (org.ovirt.thread.pool-8-thread-19) [2d37732c] FINISH, VmReplicateDiskStartVDSCommand, log id: 3f14f8ef >2014-08-11 15:55:04,359 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SyncImageGroupDataVDSCommand] (org.ovirt.thread.pool-8-thread-19) [2d37732c] START, SyncImageGroupDataVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, dstDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, syncType=INTERNAL), log id: 73eac977 >2014-08-11 15:55:04,798 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SyncImageGroupDataVDSCommand] (org.ovirt.thread.pool-8-thread-19) [2d37732c] FINISH, SyncImageGroupDataVDSCommand, log id: 73eac977 >2014-08-11 15:55:04,833 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-19) [2d37732c] CommandMultiAsyncTasks::AttachTask: Attaching task c55c14c4-091e-4219-9d11-75156cdb023b to command 3b7c02b8-97d6-4b4e-b584-056c4573183a. >2014-08-11 15:55:04,855 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-19) [2d37732c] Adding task c55c14c4-091e-4219-9d11-75156cdb023b (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:55:04,880 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-19) [2d37732c] BaseAsyncTask::startPollingTask: Starting to poll task c55c14c4-091e-4219-9d11-75156cdb023b. >2014-08-11 15:55:04,881 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-19) [2d37732c] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateDisk completed, handling the result. >2014-08-11 15:55:04,881 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-19) [2d37732c] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateDisk succeeded, clearing tasks. >2014-08-11 15:55:04,882 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-19) [2d37732c] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 0e064b4b-978c-4428-9738-b602124c6587 >2014-08-11 15:55:04,882 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-19) [2d37732c] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 0e064b4b-978c-4428-9738-b602124c6587), log id: 4ecc6797 >2014-08-11 15:55:04,884 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-19) [2d37732c] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=0e064b4b-978c-4428-9738-b602124c6587), log id: 4da512eb >2014-08-11 15:55:05,085 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-19) [2d37732c] FINISH, HSMClearTaskVDSCommand, log id: 4da512eb >2014-08-11 15:55:05,086 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-19) [2d37732c] FINISH, SPMClearTaskVDSCommand, log id: 4ecc6797 >2014-08-11 15:55:05,095 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-19) [2d37732c] BaseAsyncTask::removeTaskFromDB: Removed task 0e064b4b-978c-4428-9738-b602124c6587 from DataBase >2014-08-11 15:55:05,097 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-19) [2d37732c] [within thread]: Some of the tasks related to command id 3b7c02b8-97d6-4b4e-b584-056c4573183a were not cleared yet (Task id c55c14c4-091e-4219-9d11-75156cdb023b is in state Polling). >2014-08-11 15:55:14,137 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-44) Polling and updating Async Tasks: 3 tasks, 1 tasks to poll now >2014-08-11 15:55:14,144 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-44) SPMAsyncTask::PollTask: Polling task c55c14c4-091e-4219-9d11-75156cdb023b (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-11 15:55:14,145 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-44) Finished polling Tasks, will poll again in 10 seconds. >2014-08-11 15:56:08,584 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-89) Setting new tasks map. The map contains now 1 tasks >2014-08-11 15:56:14,181 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-11) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-11 15:56:15,600 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-11) Finished polling Tasks, will poll again in 10 seconds. >2014-08-11 15:57:00,849 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-31) SPMAsyncTask::PollTask: Polling task c55c14c4-091e-4219-9d11-75156cdb023b (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:57:00,859 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-31) BaseAsyncTask::onTaskEndSuccess: Task c55c14c4-091e-4219-9d11-75156cdb023b (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:57:00,861 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-31) CommandAsyncTask::endActionIfNecessary: All tasks of command 3b7c02b8-97d6-4b4e-b584-056c4573183a has ended -> executing endAction >2014-08-11 15:57:00,862 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-31) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 3b7c02b8-97d6-4b4e-b584-056c4573183a): calling endAction . >2014-08-11 15:57:00,863 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-9) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction LiveMigrateDisk, executionIndex: 1 >2014-08-11 15:57:00,949 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] Ending command successfully: org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand >2014-08-11 15:57:00,950 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] Running command: LiveMigrateDiskCommand Task handler: VmReplicateDiskFinishTaskHandler internal: false. Entities affected : ID: b5576e45-7f13-465f-b26f-05d52faf28d1 Type: DiskAction group CONFIGURE_DISK_STORAGE with role type USER, ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 15:57:00,977 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] START, VmReplicateDiskFinishVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327), log id: 3efaea81 >2014-08-11 15:57:01,167 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] FINISH, VmReplicateDiskFinishVDSCommand, log id: 3efaea81 >2014-08-11 15:57:01,175 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageId = d7c1ba6b-4a30-415d-ba40-1d5499786ae8), log id: 15597c5f >2014-08-11 15:57:01,203 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@a5a46199, log id: 15597c5f >2014-08-11 15:57:01,232 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageId = 5a8224b1-9aa1-4e55-b26f-2f72f91b8fcc), log id: 4f59c14e >2014-08-11 15:57:01,294 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@26aee9ef, log id: 4f59c14e >2014-08-11 15:57:01,307 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageId = 3812059c-f566-4975-955f-9937a0885b61), log id: 5b3604ee >2014-08-11 15:57:01,353 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@2466d8b4, log id: 5b3604ee >2014-08-11 15:57:01,365 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageId = 26308abd-8540-410e-8018-58cd2157084d), log id: 1d40aaf1 >2014-08-11 15:57:01,397 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@ce069bb5, log id: 1d40aaf1 >2014-08-11 15:57:01,433 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, postZeros = false, forceDelete = false), log id: 2223543e >2014-08-11 15:57:02,604 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] FINISH, DeleteImageGroupVDSCommand, log id: 2223543e >2014-08-11 15:57:02,652 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-9) [2d37732c] CommandMultiAsyncTasks::AttachTask: Attaching task c6780c9c-0748-49c0-aecc-e68af8c6c40b to command 3b7c02b8-97d6-4b4e-b584-056c4573183a. >2014-08-11 15:57:02,682 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-9) [2d37732c] Adding task c6780c9c-0748-49c0-aecc-e68af8c6c40b (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 15:57:02,707 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-9) [2d37732c] BaseAsyncTask::startPollingTask: Starting to poll task c6780c9c-0748-49c0-aecc-e68af8c6c40b. >2014-08-11 15:57:02,708 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-9) [2d37732c] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateDisk completed, handling the result. >2014-08-11 15:57:02,708 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-9) [2d37732c] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateDisk succeeded, clearing tasks. >2014-08-11 15:57:02,709 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-9) [2d37732c] SPMAsyncTask::ClearAsyncTask: Attempting to clear task c55c14c4-091e-4219-9d11-75156cdb023b >2014-08-11 15:57:02,710 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = c55c14c4-091e-4219-9d11-75156cdb023b), log id: 788d2fd9 >2014-08-11 15:57:02,713 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=c55c14c4-091e-4219-9d11-75156cdb023b), log id: 7d408d63 >2014-08-11 15:57:02,781 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] FINISH, HSMClearTaskVDSCommand, log id: 7d408d63 >2014-08-11 15:57:02,782 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-9) [2d37732c] FINISH, SPMClearTaskVDSCommand, log id: 788d2fd9 >2014-08-11 15:57:02,798 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-9) [2d37732c] BaseAsyncTask::removeTaskFromDB: Removed task c55c14c4-091e-4219-9d11-75156cdb023b from DataBase >2014-08-11 15:57:02,799 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-9) [2d37732c] [within thread]: Some of the tasks related to command id 3b7c02b8-97d6-4b4e-b584-056c4573183a were not cleared yet (Task id c6780c9c-0748-49c0-aecc-e68af8c6c40b is in state Polling). >2014-08-11 15:57:10,863 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-62) Polling and updating Async Tasks: 2 tasks, 1 tasks to poll now >2014-08-11 15:57:11,004 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-62) SPMAsyncTask::PollTask: Polling task c6780c9c-0748-49c0-aecc-e68af8c6c40b (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 15:57:11,021 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-62) BaseAsyncTask::onTaskEndSuccess: Task c6780c9c-0748-49c0-aecc-e68af8c6c40b (Parent Command LiveMigrateDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 15:57:11,023 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-62) CommandAsyncTask::endActionIfNecessary: All tasks of command 3b7c02b8-97d6-4b4e-b584-056c4573183a has ended -> executing endAction >2014-08-11 15:57:11,024 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-62) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 3b7c02b8-97d6-4b4e-b584-056c4573183a): calling endAction . >2014-08-11 15:57:11,025 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-8) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction LiveMigrateDisk, executionIndex: 2 >2014-08-11 15:57:11,095 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-8) [2d37732c] Ending command successfully: org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand >2014-08-11 15:57:11,095 INFO [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (org.ovirt.thread.pool-8-thread-8) [2d37732c] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-11 15:57:11,154 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-8) [2d37732c] Correlation ID: 2d37732c, Call Stack: null, Custom Event ID: -1, Message: User admin finished moving disk vm3_Disk1 to domain sd35_1. >2014-08-11 15:57:11,155 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-8) [2d37732c] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateDisk completed, handling the result. >2014-08-11 15:57:11,155 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-8) [2d37732c] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type LiveMigrateDisk succeeded, clearing tasks. >2014-08-11 15:57:11,163 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-8) [2d37732c] SPMAsyncTask::ClearAsyncTask: Attempting to clear task c6780c9c-0748-49c0-aecc-e68af8c6c40b >2014-08-11 15:57:11,163 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-8) [2d37732c] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = c6780c9c-0748-49c0-aecc-e68af8c6c40b), log id: 7a89f6e3 >2014-08-11 15:57:11,165 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-8) [2d37732c] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=c6780c9c-0748-49c0-aecc-e68af8c6c40b), log id: 77afefee >2014-08-11 15:57:11,239 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-8) [2d37732c] FINISH, HSMClearTaskVDSCommand, log id: 77afefee >2014-08-11 15:57:11,240 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-8) [2d37732c] FINISH, SPMClearTaskVDSCommand, log id: 7a89f6e3 >2014-08-11 15:57:11,254 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-8) [2d37732c] BaseAsyncTask::removeTaskFromDB: Removed task c6780c9c-0748-49c0-aecc-e68af8c6c40b from DataBase >2014-08-11 15:57:11,255 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-8) [2d37732c] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 3b7c02b8-97d6-4b4e-b584-056c4573183a >2014-08-11 15:57:14,132 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-75) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: 15cbaa7 >2014-08-11 15:57:14,144 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-75) FINISH, FullListVdsCommand, return: [{displaySecurePort=5900, kvmEnable=true, nicModel=rtl8139,pv, guestIPs=, keyboardLayout=en-us, displayIp=10.35.0.105, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, username=Unknown, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, status=Up, emulatedMachine=pc-1.0, vmType=kvm, memGuaranteedSize=1024, display=qxl, pid=25652, smartcardEnable=false, bootMenuEnable=false, numaTune={mode=interleave}, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, guestFQDN=, smpCoresPerSocket=1, maxVCpus=160, clientIp=, devices=[Ljava.lang.Object;@6ee22aa3, vmName=vm3, fileTransferEnable=true, cpuType=SandyBridge}], log id: 15cbaa7 >2014-08-11 15:57:14,149 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Received a memballoon Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 15:57:14,150 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Received a spice Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5900, type=graphics} >2014-08-11 15:58:09,145 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-19) Setting new tasks map. The map contains now 1 tasks >2014-08-11 15:58:39,147 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-11) Setting new tasks map. The map contains now 0 tasks >2014-08-11 15:58:39,147 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-11) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 16:28:38,622 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-27) Principal admin::internal synchronized >2014-08-11 16:28:38,794 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) Attempting to update VMs/Templates Ovf. >2014-08-11 16:28:39,012 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-32) [17fa27e3] Lock Acquired to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-11 16:28:39,015 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-32) [17fa27e3] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-11 16:28:39,073 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [17fa27e3] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 78572ae5 >2014-08-11 16:28:39,074 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [17fa27e3] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-11 16:28:39,075 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [17fa27e3] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Mon Aug 11 15:13:16 IDT 2014"} > >2014-08-11 16:28:39,176 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [17fa27e3] FINISH, SetVolumeDescriptionVDSCommand, log id: 78572ae5 >2014-08-11 16:28:39,239 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-11 16:28:39,245 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 24a8d861 >2014-08-11 16:28:39,246 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] -- executeVdsBrokerCommand: >2014-08-11 16:28:39,247 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > size=71,680 > >2014-08-11 16:28:40,885 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] FINISH, UploadStreamVDSCommand, log id: 24a8d861 >2014-08-11 16:28:40,895 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 9194a5d1-b56f-4b83-b47d-5d170cf92e93 >2014-08-11 16:28:40,896 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] CommandMultiAsyncTasks::AttachTask: Attaching task 662974f8-c0cf-4c1f-93b1-14f4a8483cb1 to command 9194a5d1-b56f-4b83-b47d-5d170cf92e93. >2014-08-11 16:28:40,929 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] Adding task 662974f8-c0cf-4c1f-93b1-14f4a8483cb1 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 16:28:40,945 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-11 16:28:40,950 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 7e4d9e0a >2014-08-11 16:28:40,951 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-11 16:28:40,951 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Mon Aug 11 16:28:39 IDT 2014","Size":71680} > >2014-08-11 16:28:41,059 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] FINISH, SetVolumeDescriptionVDSCommand, log id: 7e4d9e0a >2014-08-11 16:28:41,073 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 17243703 >2014-08-11 16:28:41,075 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-11 16:28:41,076 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Mon Aug 11 15:13:16 IDT 2014"} > >2014-08-11 16:28:41,166 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [2d1d3fa5] FINISH, SetVolumeDescriptionVDSCommand, log id: 17243703 >2014-08-11 16:28:41,211 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-32) [4337c859] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-11 16:28:41,213 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-32) [4337c859] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 681b7b88 >2014-08-11 16:28:41,214 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-32) [4337c859] -- executeVdsBrokerCommand: >2014-08-11 16:28:41,214 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-32) [4337c859] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > size=71,680 > >2014-08-11 16:28:43,080 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-32) [4337c859] FINISH, UploadStreamVDSCommand, log id: 681b7b88 >2014-08-11 16:28:43,082 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-32) [4337c859] CommandMultiAsyncTasks::AttachTask: Attaching task 0dcbd4b7-f367-4d0f-b61b-73fd7ac31fdf to command 9194a5d1-b56f-4b83-b47d-5d170cf92e93. >2014-08-11 16:28:43,132 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-32) [4337c859] Adding task 0dcbd4b7-f367-4d0f-b61b-73fd7ac31fdf (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 16:28:43,182 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-32) [4337c859] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-11 16:28:43,183 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-32) [4337c859] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-11 16:28:43,184 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-32) [4337c859] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-11 16:28:43,185 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-32) [4337c859] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-11 16:28:43,186 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-32) [4337c859] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-11 16:28:43,189 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [4337c859] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 1b412dfb >2014-08-11 16:28:43,190 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [4337c859] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-11 16:28:43,191 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [4337c859] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Mon Aug 11 16:28:39 IDT 2014","Size":71680} > >2014-08-11 16:28:43,490 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-32) [4337c859] FINISH, SetVolumeDescriptionVDSCommand, log id: 1b412dfb >2014-08-11 16:28:43,497 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-32) [4337c859] BaseAsyncTask::startPollingTask: Starting to poll task 662974f8-c0cf-4c1f-93b1-14f4a8483cb1. >2014-08-11 16:28:43,498 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-32) [4337c859] BaseAsyncTask::startPollingTask: Starting to poll task 0dcbd4b7-f367-4d0f-b61b-73fd7ac31fdf. >2014-08-11 16:28:51,092 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-70) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 16:28:51,147 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-70) SPMAsyncTask::PollTask: Polling task 662974f8-c0cf-4c1f-93b1-14f4a8483cb1 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 16:28:51,149 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-70) BaseAsyncTask::onTaskEndSuccess: Task 662974f8-c0cf-4c1f-93b1-14f4a8483cb1 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 16:28:51,150 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-70) Task with DB Task ID e3054941-c2bc-4ed4-a134-f63071a4c23d and VDSM Task ID 0dcbd4b7-f367-4d0f-b61b-73fd7ac31fdf is in state Polling. End action for command 9194a5d1-b56f-4b83-b47d-5d170cf92e93 will proceed when all the entitys tasks are completed. >2014-08-11 16:28:51,152 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-70) SPMAsyncTask::PollTask: Polling task 0dcbd4b7-f367-4d0f-b61b-73fd7ac31fdf (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 16:28:51,153 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-70) BaseAsyncTask::onTaskEndSuccess: Task 0dcbd4b7-f367-4d0f-b61b-73fd7ac31fdf (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 16:28:51,155 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-70) CommandAsyncTask::endActionIfNecessary: All tasks of command 9194a5d1-b56f-4b83-b47d-5d170cf92e93 has ended -> executing endAction >2014-08-11 16:28:51,156 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-70) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 9194a5d1-b56f-4b83-b47d-5d170cf92e93): calling endAction . >2014-08-11 16:28:51,157 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-37) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-08-11 16:28:51,180 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-08-11 16:28:51,181 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-11 16:28:51,182 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-11 16:28:51,183 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-11 16:28:51,184 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-11 16:28:51,185 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-11 16:28:51,186 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-08-11 16:28:51,187 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-08-11 16:28:51,188 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 662974f8-c0cf-4c1f-93b1-14f4a8483cb1 >2014-08-11 16:28:51,188 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 662974f8-c0cf-4c1f-93b1-14f4a8483cb1), log id: 54517db2 >2014-08-11 16:28:51,190 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=662974f8-c0cf-4c1f-93b1-14f4a8483cb1), log id: 32135c5d >2014-08-11 16:28:51,262 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] FINISH, HSMClearTaskVDSCommand, log id: 32135c5d >2014-08-11 16:28:51,263 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] FINISH, SPMClearTaskVDSCommand, log id: 54517db2 >2014-08-11 16:28:51,308 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] BaseAsyncTask::removeTaskFromDB: Removed task 662974f8-c0cf-4c1f-93b1-14f4a8483cb1 from DataBase >2014-08-11 16:28:51,309 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 0dcbd4b7-f367-4d0f-b61b-73fd7ac31fdf >2014-08-11 16:28:51,310 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 0dcbd4b7-f367-4d0f-b61b-73fd7ac31fdf), log id: 787a7eab >2014-08-11 16:28:51,313 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=0dcbd4b7-f367-4d0f-b61b-73fd7ac31fdf), log id: 32124a58 >2014-08-11 16:28:51,441 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] FINISH, HSMClearTaskVDSCommand, log id: 32124a58 >2014-08-11 16:28:51,442 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] FINISH, SPMClearTaskVDSCommand, log id: 787a7eab >2014-08-11 16:28:51,457 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] BaseAsyncTask::removeTaskFromDB: Removed task 0dcbd4b7-f367-4d0f-b61b-73fd7ac31fdf from DataBase >2014-08-11 16:28:51,458 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-37) [17fa27e3] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 9194a5d1-b56f-4b83-b47d-5d170cf92e93 >2014-08-11 16:30:09,175 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-84) Setting new tasks map. The map contains now 0 tasks >2014-08-11 16:30:09,176 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-84) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 16:54:26,584 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-8-thread-22) Executing a command: java.util.concurrent.FutureTask , but note that there are 0 tasks in the queue. >2014-08-11 17:28:38,714 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-35) Principal admin::internal synchronized >2014-08-11 17:28:43,500 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Attempting to update VMs/Templates Ovf. >2014-08-11 17:33:37,210 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (http--0.0.0.0-8080-2) [f91a44b] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group CONNECT_TO_VM with role type USER >2014-08-11 17:33:37,256 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-2) [f91a44b] START, SetVmTicketVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, ticket=UVCr/jABXSCW, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 45d3a5aa >2014-08-11 17:33:37,269 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-2) [f91a44b] FINISH, SetVmTicketVDSCommand, log id: 45d3a5aa >2014-08-11 17:33:37,431 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [f91a44b] Correlation ID: f91a44b, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM vm3 >2014-08-11 17:33:51,581 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (http--0.0.0.0-8080-2) [d9573e1] Running command: SetVmTicketCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group CONNECT_TO_VM with role type USER >2014-08-11 17:33:51,585 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-2) [d9573e1] START, SetVmTicketVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, ticket=U5TeM7wKpv4h, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 5e16c873 >2014-08-11 17:33:51,637 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-2) [d9573e1] FINISH, SetVmTicketVDSCommand, log id: 5e16c873 >2014-08-11 17:33:51,674 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [d9573e1] Correlation ID: d9573e1, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM vm4 >2014-08-11 17:34:37,833 WARN [org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil] (org.ovirt.thread.pool-8-thread-44) Executing a command: java.util.concurrent.FutureTask , but note that there are 1 tasks in the queue. >2014-08-11 17:34:45,366 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (http--0.0.0.0-8080-2) [274f78c4] Running command: SetVmTicketCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group CONNECT_TO_VM with role type USER >2014-08-11 17:34:45,369 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-2) [274f78c4] START, SetVmTicketVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, ticket=KmTZcZ4KGlGO, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 69cb7e78 >2014-08-11 17:34:45,381 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-2) [274f78c4] FINISH, SetVmTicketVDSCommand, log id: 69cb7e78 >2014-08-11 17:34:45,475 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [274f78c4] Correlation ID: 274f78c4, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM vm4 >2014-08-11 17:39:54,084 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [10126e78] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-11 17:39:54,431 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-17) [10126e78] Running command: HibernateVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 17:39:54,529 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-17) [10126e78] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@a627d0cc), log id: 7a3da23 >2014-08-11 17:39:54,531 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-17) [10126e78] FINISH, UpdateVmDynamicDataVDSCommand, log id: 7a3da23 >2014-08-11 17:39:54,568 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-17) [10126e78] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 4c2b3c72-a901-435d-9a9f-8628b2263c7e, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = bdca8406-da33-4323-9fa9-53d245f0a0ed, newImageDescription = ), log id: 4c152031 >2014-08-11 17:39:54,570 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-17) [10126e78] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 17:39:55,015 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-17) [10126e78] FINISH, CreateImageVDSCommand, return: bdca8406-da33-4323-9fa9-53d245f0a0ed, log id: 4c152031 >2014-08-11 17:39:55,232 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-17) [10126e78] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 9917d9ae-55b8-44be-86d7-60e840c1fdcc >2014-08-11 17:39:55,232 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-17) [10126e78] CommandMultiAsyncTasks::AttachTask: Attaching task fd387023-cd7f-4ba1-9d20-6d42f853109f to command 9917d9ae-55b8-44be-86d7-60e840c1fdcc. >2014-08-11 17:39:55,253 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-17) [10126e78] Adding task fd387023-cd7f-4ba1-9d20-6d42f853109f (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 17:39:55,276 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-17) [10126e78] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = cdc5a7e1-4d16-404d-90a4-c0ed6584c11b, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = 269bcc1a-8426-49ae-a837-e9653759fcc8, newImageDescription = ), log id: 2eb7ced2 >2014-08-11 17:39:55,278 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-17) [10126e78] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 17:39:55,887 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-17) [10126e78] FINISH, CreateImageVDSCommand, return: 269bcc1a-8426-49ae-a837-e9653759fcc8, log id: 2eb7ced2 >2014-08-11 17:39:55,948 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-17) [10126e78] CommandMultiAsyncTasks::AttachTask: Attaching task ae83c418-616b-4224-8a8e-87057b8a90c2 to command 9917d9ae-55b8-44be-86d7-60e840c1fdcc. >2014-08-11 17:39:55,996 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-17) [10126e78] Adding task ae83c418-616b-4224-8a8e-87057b8a90c2 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 17:39:56,006 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-17) [10126e78] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@ff3da51a), log id: 2d5b97d8 >2014-08-11 17:39:56,013 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-17) [10126e78] FINISH, UpdateVmDynamicDataVDSCommand, log id: 2d5b97d8 >2014-08-11 17:39:56,121 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-17) [10126e78] Correlation ID: 10126e78, Job ID: 3feb2b3a-50b5-4d54-96a1-1f9b81b647fe, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm3 was initiated by User admin (Host: h1). >2014-08-11 17:39:56,122 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-17) [10126e78] BaseAsyncTask::startPollingTask: Starting to poll task fd387023-cd7f-4ba1-9d20-6d42f853109f. >2014-08-11 17:39:56,123 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-17) [10126e78] BaseAsyncTask::startPollingTask: Starting to poll task ae83c418-616b-4224-8a8e-87057b8a90c2. >2014-08-11 17:39:56,123 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-17) [10126e78] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-11 17:39:57,589 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-50) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 17:39:57,597 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-50) SPMAsyncTask::PollTask: Polling task ae83c418-616b-4224-8a8e-87057b8a90c2 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-11 17:39:57,599 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-50) SPMAsyncTask::PollTask: Polling task fd387023-cd7f-4ba1-9d20-6d42f853109f (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 17:39:57,678 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-50) BaseAsyncTask::onTaskEndSuccess: Task fd387023-cd7f-4ba1-9d20-6d42f853109f (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 17:39:57,679 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-50) Task with DB Task ID 3c8a8e9c-c1d3-40dc-88b3-71364b57016e and VDSM Task ID ae83c418-616b-4224-8a8e-87057b8a90c2 is in state Polling. End action for command 9917d9ae-55b8-44be-86d7-60e840c1fdcc will proceed when all the entitys tasks are completed. >2014-08-11 17:39:57,681 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-50) Finished polling Tasks, will poll again in 10 seconds. >2014-08-11 17:39:58,125 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from PreparingForHibernate --> Up >2014-08-11 17:40:07,690 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-62) SPMAsyncTask::PollTask: Polling task ae83c418-616b-4224-8a8e-87057b8a90c2 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 17:40:07,731 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-62) BaseAsyncTask::onTaskEndSuccess: Task ae83c418-616b-4224-8a8e-87057b8a90c2 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 17:40:07,732 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-62) CommandAsyncTask::endActionIfNecessary: All tasks of command 9917d9ae-55b8-44be-86d7-60e840c1fdcc has ended -> executing endAction >2014-08-11 17:40:07,732 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-62) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 9917d9ae-55b8-44be-86d7-60e840c1fdcc): calling endAction . >2014-08-11 17:40:07,733 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-44) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-11 17:40:07,773 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-44) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-11 17:40:07,796 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-44) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,4c2b3c72-a901-435d-9a9f-8628b2263c7e,bdca8406-da33-4323-9fa9-53d245f0a0ed,cdc5a7e1-4d16-404d-90a4-c0ed6584c11b,269bcc1a-8426-49ae-a837-e9653759fcc8), log id: 4efd48cc >2014-08-11 17:40:07,814 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-44) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,4c2b3c72-a901-435d-9a9f-8628b2263c7e,bdca8406-da33-4323-9fa9-53d245f0a0ed,cdc5a7e1-4d16-404d-90a4-c0ed6584c11b,269bcc1a-8426-49ae-a837-e9653759fcc8), log id: 77d6c636 >2014-08-11 17:40:07,820 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-44) FINISH, HibernateBrokerVDSCommand, log id: 77d6c636 >2014-08-11 17:40:07,847 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-44) FINISH, HibernateVDSCommand, log id: 4efd48cc >2014-08-11 17:40:07,848 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-44) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-11 17:40:07,848 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-44) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-11 17:40:07,856 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-44) SPMAsyncTask::ClearAsyncTask: Attempting to clear task ae83c418-616b-4224-8a8e-87057b8a90c2 >2014-08-11 17:40:07,857 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-44) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = ae83c418-616b-4224-8a8e-87057b8a90c2), log id: 7c9c0b7e >2014-08-11 17:40:07,859 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-44) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=ae83c418-616b-4224-8a8e-87057b8a90c2), log id: 68b6bd1b >2014-08-11 17:40:07,946 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-44) FINISH, HSMClearTaskVDSCommand, log id: 68b6bd1b >2014-08-11 17:40:07,947 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-44) FINISH, SPMClearTaskVDSCommand, log id: 7c9c0b7e >2014-08-11 17:40:07,955 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-44) BaseAsyncTask::removeTaskFromDB: Removed task ae83c418-616b-4224-8a8e-87057b8a90c2 from DataBase >2014-08-11 17:40:07,955 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-44) SPMAsyncTask::ClearAsyncTask: Attempting to clear task fd387023-cd7f-4ba1-9d20-6d42f853109f >2014-08-11 17:40:07,956 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-44) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = fd387023-cd7f-4ba1-9d20-6d42f853109f), log id: 550dc950 >2014-08-11 17:40:07,957 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-44) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=fd387023-cd7f-4ba1-9d20-6d42f853109f), log id: 7c1bcecf >2014-08-11 17:40:08,027 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-44) FINISH, HSMClearTaskVDSCommand, log id: 7c1bcecf >2014-08-11 17:40:08,027 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-44) FINISH, SPMClearTaskVDSCommand, log id: 550dc950 >2014-08-11 17:40:08,046 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-44) BaseAsyncTask::removeTaskFromDB: Removed task fd387023-cd7f-4ba1-9d20-6d42f853109f from DataBase >2014-08-11 17:40:08,046 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-44) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 9917d9ae-55b8-44be-86d7-60e840c1fdcc >2014-08-11 17:40:10,432 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from SavingState --> Up >2014-08-11 17:41:17,416 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-67) Setting new tasks map. The map contains now 0 tasks >2014-08-11 17:41:17,417 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-67) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 17:43:49,809 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [49207b88] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 17:43:49,938 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-14) [49207b88] Running command: HibernateVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 17:43:49,961 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-14) [49207b88] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@f15f992), log id: 6fc6b48e >2014-08-11 17:43:49,963 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-14) [49207b88] FINISH, UpdateVmDynamicDataVDSCommand, log id: 6fc6b48e >2014-08-11 17:43:49,976 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-14) [49207b88] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = a8e37c70-8fa4-46f7-bc12-f7b02e5b5856, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = 041ad999-7aa9-4dcf-a3b1-bf962cc75c25, newImageDescription = ), log id: 5ac294aa >2014-08-11 17:43:49,979 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-14) [49207b88] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 17:43:50,411 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-14) [49207b88] FINISH, CreateImageVDSCommand, return: 041ad999-7aa9-4dcf-a3b1-bf962cc75c25, log id: 5ac294aa >2014-08-11 17:43:50,518 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) [49207b88] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 0798f767-b597-48ae-8558-6c1d4d5483fb >2014-08-11 17:43:50,519 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-14) [49207b88] CommandMultiAsyncTasks::AttachTask: Attaching task 9c2a5d70-29fe-437b-b8a7-45f5efd4b79a to command 0798f767-b597-48ae-8558-6c1d4d5483fb. >2014-08-11 17:43:50,527 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-14) [49207b88] Adding task 9c2a5d70-29fe-437b-b8a7-45f5efd4b79a (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 17:43:50,542 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-14) [49207b88] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 812042c3-6adc-4cf2-aaf4-e3f90efd5e5f, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = cd41aaf7-4ca5-4c68-bbdc-1ae1690705ff, newImageDescription = ), log id: 6af7c77 >2014-08-11 17:43:50,544 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-14) [49207b88] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 17:43:51,209 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-14) [49207b88] FINISH, CreateImageVDSCommand, return: cd41aaf7-4ca5-4c68-bbdc-1ae1690705ff, log id: 6af7c77 >2014-08-11 17:43:51,242 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-14) [49207b88] CommandMultiAsyncTasks::AttachTask: Attaching task d1919e29-2350-4faa-8241-34d8964ddb72 to command 0798f767-b597-48ae-8558-6c1d4d5483fb. >2014-08-11 17:43:51,263 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-14) [49207b88] Adding task d1919e29-2350-4faa-8241-34d8964ddb72 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 17:43:51,274 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-14) [49207b88] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@ed232d17), log id: 7fd71f2b >2014-08-11 17:43:51,279 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-14) [49207b88] FINISH, UpdateVmDynamicDataVDSCommand, log id: 7fd71f2b >2014-08-11 17:43:51,305 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-14) [49207b88] Correlation ID: 49207b88, Job ID: c99c4197-9b8a-4772-bab4-a2bb308134b6, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm was initiated by User admin (Host: h1). >2014-08-11 17:43:51,306 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) [49207b88] BaseAsyncTask::startPollingTask: Starting to poll task 9c2a5d70-29fe-437b-b8a7-45f5efd4b79a. >2014-08-11 17:43:51,307 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) [49207b88] BaseAsyncTask::startPollingTask: Starting to poll task d1919e29-2350-4faa-8241-34d8964ddb72. >2014-08-11 17:43:51,307 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-14) [49207b88] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 17:43:57,738 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-79) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 17:43:57,746 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) SPMAsyncTask::PollTask: Polling task 9c2a5d70-29fe-437b-b8a7-45f5efd4b79a (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 17:43:57,798 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) BaseAsyncTask::onTaskEndSuccess: Task 9c2a5d70-29fe-437b-b8a7-45f5efd4b79a (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 17:43:57,799 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-79) Task with DB Task ID 1d655f8d-4ff3-4f37-9971-927c78ac38fb and VDSM Task ID d1919e29-2350-4faa-8241-34d8964ddb72 is in state Polling. End action for command 0798f767-b597-48ae-8558-6c1d4d5483fb will proceed when all the entitys tasks are completed. >2014-08-11 17:43:57,799 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) SPMAsyncTask::PollTask: Polling task d1919e29-2350-4faa-8241-34d8964ddb72 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 17:43:57,807 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) BaseAsyncTask::onTaskEndSuccess: Task d1919e29-2350-4faa-8241-34d8964ddb72 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 17:43:57,807 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-79) CommandAsyncTask::endActionIfNecessary: All tasks of command 0798f767-b597-48ae-8558-6c1d4d5483fb has ended -> executing endAction >2014-08-11 17:43:57,807 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-79) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 0798f767-b597-48ae-8558-6c1d4d5483fb): calling endAction . >2014-08-11 17:43:57,808 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-43) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-11 17:43:57,881 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-43) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-11 17:43:57,883 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-43) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,a8e37c70-8fa4-46f7-bc12-f7b02e5b5856,041ad999-7aa9-4dcf-a3b1-bf962cc75c25,812042c3-6adc-4cf2-aaf4-e3f90efd5e5f,cd41aaf7-4ca5-4c68-bbdc-1ae1690705ff), log id: 2130d180 >2014-08-11 17:43:57,884 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-43) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,a8e37c70-8fa4-46f7-bc12-f7b02e5b5856,041ad999-7aa9-4dcf-a3b1-bf962cc75c25,812042c3-6adc-4cf2-aaf4-e3f90efd5e5f,cd41aaf7-4ca5-4c68-bbdc-1ae1690705ff), log id: 6a8d701d >2014-08-11 17:43:57,890 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-43) FINISH, HibernateBrokerVDSCommand, log id: 6a8d701d >2014-08-11 17:43:57,898 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-43) FINISH, HibernateVDSCommand, log id: 2130d180 >2014-08-11 17:43:57,899 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-43) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-11 17:43:57,899 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-43) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-11 17:43:57,906 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-43) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 9c2a5d70-29fe-437b-b8a7-45f5efd4b79a >2014-08-11 17:43:57,907 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 9c2a5d70-29fe-437b-b8a7-45f5efd4b79a), log id: 86240f4 >2014-08-11 17:43:57,909 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=9c2a5d70-29fe-437b-b8a7-45f5efd4b79a), log id: 6a965778 >2014-08-11 17:43:58,020 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) FINISH, HSMClearTaskVDSCommand, log id: 6a965778 >2014-08-11 17:43:58,020 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) FINISH, SPMClearTaskVDSCommand, log id: 86240f4 >2014-08-11 17:43:58,030 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-43) BaseAsyncTask::removeTaskFromDB: Removed task 9c2a5d70-29fe-437b-b8a7-45f5efd4b79a from DataBase >2014-08-11 17:43:58,031 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-43) SPMAsyncTask::ClearAsyncTask: Attempting to clear task d1919e29-2350-4faa-8241-34d8964ddb72 >2014-08-11 17:43:58,031 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = d1919e29-2350-4faa-8241-34d8964ddb72), log id: 7fdc389e >2014-08-11 17:43:58,033 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=d1919e29-2350-4faa-8241-34d8964ddb72), log id: 53094ea6 >2014-08-11 17:43:58,120 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) FINISH, HSMClearTaskVDSCommand, log id: 53094ea6 >2014-08-11 17:43:58,121 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) FINISH, SPMClearTaskVDSCommand, log id: 7fdc389e >2014-08-11 17:43:58,130 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-43) BaseAsyncTask::removeTaskFromDB: Removed task d1919e29-2350-4faa-8241-34d8964ddb72 from DataBase >2014-08-11 17:43:58,130 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-43) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 0798f767-b597-48ae-8558-6c1d4d5483fb >2014-08-11 17:43:59,905 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-84) START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, force=false, secondsToWait=0, gracefully=false, reason=), log id: 9fb3a40 >2014-08-11 17:43:59,916 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-84) FINISH, DestroyVDSCommand, log id: 9fb3a40 >2014-08-11 17:43:59,936 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-84) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm on Host h1 is suspended. >2014-08-11 17:43:59,938 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) VM vm (14c0fa91-1814-438b-8d7a-4e7de1e1da25) is running in db and not running in VDS h1 >2014-08-11 17:43:59,938 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-84) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 59d25d59 >2014-08-11 17:43:59,942 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-84) FINISH, FullListVdsCommand, return: [], log id: 59d25d59 >2014-08-11 17:43:59,974 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-18) [6cb5f185] Running command: ProcessDownVmCommand internal: true. >2014-08-11 17:45:17,421 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-14) Setting new tasks map. The map contains now 0 tasks >2014-08-11 17:45:17,422 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-14) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 17:48:59,317 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [16f312ca] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 17:48:59,551 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [16f312ca] START, IsVmDuringInitiatingVDSCommand( vmId = 14c0fa91-1814-438b-8d7a-4e7de1e1da25), log id: 13660337 >2014-08-11 17:48:59,551 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [16f312ca] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 13660337 >2014-08-11 17:48:59,667 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-25) [16f312ca] Running command: RunVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 17:48:59,765 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-25) [16f312ca] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@9a12508f), log id: 2d1afc5c >2014-08-11 17:48:59,774 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-25) [16f312ca] FINISH, UpdateVmDynamicDataVDSCommand, log id: 2d1afc5c >2014-08-11 17:48:59,795 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-25) [16f312ca] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, vm=VM [vm]), log id: 28c27818 >2014-08-11 17:48:59,843 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-25) [16f312ca] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, vm=VM [vm]), log id: 26db859 >2014-08-11 17:48:59,937 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-25) [16f312ca] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,hiberVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,a8e37c70-8fa4-46f7-bc12-f7b02e5b5856,041ad999-7aa9-4dcf-a3b1-bf962cc75c25,812042c3-6adc-4cf2-aaf4-e3f90efd5e5f,cd41aaf7-4ca5-4c68-bbdc-1ae1690705ff,transparentHugePages=true,vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=68e21ff2-4b5a-4b29-8765-eec79f1b15dd}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=496deb19-46c7-4b58-94a3-06b7fa34f7ac}, {shared=false, index=0, volumeID=3f8b5bbe-741b-4a4f-9397-84123d530772, propagateErrors=off, format=raw, type=disk, iface=virtio, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, imageID=99e5cda5-262b-4469-b336-04e3f0cf7c38, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=99e5cda5-262b-4469-b336-04e3f0cf7c38}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:57, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=1e5b18b9-16eb-4b96-83b6-4568b8bed355}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=3959560b-27d1-4acf-acd9-c39f0c4148b2}],vmName=vm,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 17:49:00,042 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-25) [16f312ca] FINISH, CreateVDSCommand, log id: 26db859 >2014-08-11 17:49:00,081 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-25) [16f312ca] FINISH, CreateVmVDSCommand, return: RestoringState, log id: 28c27818 >2014-08-11 17:49:00,081 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-25) [16f312ca] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 17:49:00,098 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-25) [16f312ca] Correlation ID: 16f312ca, Job ID: a7f39d6b-c6ff-442f-a638-62bca36c5f04, Call Stack: null, Custom Event ID: -1, Message: VM vm was started by admin (Host: h1). >2014-08-11 17:49:01,714 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) [6c840493] VM vm 14c0fa91-1814-438b-8d7a-4e7de1e1da25 moved from RestoringState --> Up >2014-08-11 17:49:01,715 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-23) [6c840493] START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 4bfe2f20 >2014-08-11 17:49:01,725 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-23) [6c840493] FINISH, FullListVdsCommand, return: [{displaySecurePort=5902, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=22424, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@1cc804da, vmName=vm, fileTransferEnable=true, cpuType=SandyBridge}], log id: 4bfe2f20 >2014-08-11 17:49:01,728 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) [6c840493] Received a memballoon Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 17:49:01,730 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) [6c840493] Received a spice Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5902, type=graphics} >2014-08-11 17:49:01,854 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [6c840493] Correlation ID: 16f312ca, Job ID: a7f39d6b-c6ff-442f-a638-62bca36c5f04, Call Stack: null, Custom Event ID: -1, Message: VM vm started on Host h1 >2014-08-11 17:49:01,864 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (DefaultQuartzScheduler_Worker-23) [1aa273ac] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-11 17:49:01,979 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-23) [1aa273ac] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = a8e37c70-8fa4-46f7-bc12-f7b02e5b5856, postZeros = false, forceDelete = false), log id: 4c937e24 >2014-08-11 17:49:02,506 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-23) [1aa273ac] FINISH, DeleteImageGroupVDSCommand, log id: 4c937e24 >2014-08-11 17:49:02,571 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-23) [1aa273ac] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 8caa65a6-3cf4-433a-ae78-15c6bfab9aef >2014-08-11 17:49:02,572 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-23) [1aa273ac] CommandMultiAsyncTasks::AttachTask: Attaching task 924b5ab1-5479-4d41-a5b3-e436dbbee96c to command 8caa65a6-3cf4-433a-ae78-15c6bfab9aef. >2014-08-11 17:49:02,600 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-23) [1aa273ac] Adding task 924b5ab1-5479-4d41-a5b3-e436dbbee96c (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 17:49:02,624 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-23) [1aa273ac] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 812042c3-6adc-4cf2-aaf4-e3f90efd5e5f, postZeros = false, forceDelete = false), log id: 38f818af >2014-08-11 17:49:03,091 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-23) [1aa273ac] FINISH, DeleteImageGroupVDSCommand, log id: 38f818af >2014-08-11 17:49:03,205 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-23) [1aa273ac] CommandMultiAsyncTasks::AttachTask: Attaching task 28435af6-8b8c-43f9-9d73-5ff28ee984c4 to command 8caa65a6-3cf4-433a-ae78-15c6bfab9aef. >2014-08-11 17:49:03,245 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-23) [1aa273ac] Adding task 28435af6-8b8c-43f9-9d73-5ff28ee984c4 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 17:49:03,272 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-23) [1aa273ac] BaseAsyncTask::startPollingTask: Starting to poll task 924b5ab1-5479-4d41-a5b3-e436dbbee96c. >2014-08-11 17:49:03,273 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-23) [1aa273ac] BaseAsyncTask::startPollingTask: Starting to poll task 28435af6-8b8c-43f9-9d73-5ff28ee984c4. >2014-08-11 17:49:03,274 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-23) [1aa273ac] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@b6cd96ba), log id: 626c1063 >2014-08-11 17:49:03,287 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-23) [1aa273ac] FINISH, UpdateVmDynamicDataVDSCommand, log id: 626c1063 >2014-08-11 17:49:07,363 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [66b06d7d] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 17:49:07,364 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [66b06d7d] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:255) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 17:49:07,447 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [66b06d7d] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 17:49:07,817 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-43) task id 28435af6-8b8c-43f9-9d73-5ff28ee984c4 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:49:07,817 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-43) task id 924b5ab1-5479-4d41-a5b3-e436dbbee96c is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:49:09,329 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-39) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 536298d5 >2014-08-11 17:49:09,339 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-39) FINISH, FullListVdsCommand, return: [{displaySecurePort=5902, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=22424, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@5483a3a5, vmName=vm, fileTransferEnable=true, cpuType=SandyBridge}], log id: 536298d5 >2014-08-11 17:49:09,342 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Received a memballoon Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 17:49:09,343 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Received a spice Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5902, type=graphics} >2014-08-11 17:49:17,818 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-53) task id 28435af6-8b8c-43f9-9d73-5ff28ee984c4 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:49:17,820 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-53) task id 924b5ab1-5479-4d41-a5b3-e436dbbee96c is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:49:18,019 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [2d17e3c8] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 17:49:18,020 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [2d17e3c8] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:255) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 17:49:18,048 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [2d17e3c8] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 17:49:27,821 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-75) task id 28435af6-8b8c-43f9-9d73-5ff28ee984c4 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:49:27,823 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-75) task id 924b5ab1-5479-4d41-a5b3-e436dbbee96c is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:49:37,824 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-67) task id 28435af6-8b8c-43f9-9d73-5ff28ee984c4 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:49:37,826 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-67) task id 924b5ab1-5479-4d41-a5b3-e436dbbee96c is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:49:47,827 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) task id 28435af6-8b8c-43f9-9d73-5ff28ee984c4 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:49:47,829 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) task id 924b5ab1-5479-4d41-a5b3-e436dbbee96c is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:49:57,831 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-29) task id 28435af6-8b8c-43f9-9d73-5ff28ee984c4 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:49:57,832 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-29) task id 924b5ab1-5479-4d41-a5b3-e436dbbee96c is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:50:07,835 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) task id 28435af6-8b8c-43f9-9d73-5ff28ee984c4 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:50:07,836 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) task id 28435af6-8b8c-43f9-9d73-5ff28ee984c4 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:50:07,838 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) task id 924b5ab1-5479-4d41-a5b3-e436dbbee96c has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:50:07,839 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-34) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 17:50:07,840 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) task id 28435af6-8b8c-43f9-9d73-5ff28ee984c4 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:50:07,841 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) task id 924b5ab1-5479-4d41-a5b3-e436dbbee96c has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:50:07,850 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) task id 28435af6-8b8c-43f9-9d73-5ff28ee984c4 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:50:07,851 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) SPMAsyncTask::PollTask: Polling task 28435af6-8b8c-43f9-9d73-5ff28ee984c4 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 17:50:07,876 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) BaseAsyncTask::onTaskEndSuccess: Task 28435af6-8b8c-43f9-9d73-5ff28ee984c4 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 17:50:07,877 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-34) Task with DB Task ID e5df9fbf-a6d4-44b8-96db-b0ee1e6c6787 and VDSM Task ID 924b5ab1-5479-4d41-a5b3-e436dbbee96c is in state Polling. End action for command 8caa65a6-3cf4-433a-ae78-15c6bfab9aef will proceed when all the entitys tasks are completed. >2014-08-11 17:50:07,877 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) task id 924b5ab1-5479-4d41-a5b3-e436dbbee96c has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 17:50:07,878 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) SPMAsyncTask::PollTask: Polling task 924b5ab1-5479-4d41-a5b3-e436dbbee96c (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 17:50:07,884 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) BaseAsyncTask::onTaskEndSuccess: Task 924b5ab1-5479-4d41-a5b3-e436dbbee96c (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 17:50:07,885 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-34) CommandAsyncTask::endActionIfNecessary: All tasks of command 8caa65a6-3cf4-433a-ae78-15c6bfab9aef has ended -> executing endAction >2014-08-11 17:50:07,886 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-34) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 8caa65a6-3cf4-433a-ae78-15c6bfab9aef): calling endAction . >2014-08-11 17:50:07,886 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-43) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RunVm, executionIndex: 0 >2014-08-11 17:50:07,901 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-43) Ending command successfully: org.ovirt.engine.core.bll.RunVmCommand >2014-08-11 17:50:07,921 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-43) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm completed, handling the result. >2014-08-11 17:50:07,921 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-43) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm succeeded, clearing tasks. >2014-08-11 17:50:07,934 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-43) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 924b5ab1-5479-4d41-a5b3-e436dbbee96c >2014-08-11 17:50:07,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 924b5ab1-5479-4d41-a5b3-e436dbbee96c), log id: 2dbe7922 >2014-08-11 17:50:07,938 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=924b5ab1-5479-4d41-a5b3-e436dbbee96c), log id: 3402bea4 >2014-08-11 17:50:08,021 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) FINISH, HSMClearTaskVDSCommand, log id: 3402bea4 >2014-08-11 17:50:08,021 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) FINISH, SPMClearTaskVDSCommand, log id: 2dbe7922 >2014-08-11 17:50:08,033 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-43) BaseAsyncTask::removeTaskFromDB: Removed task 924b5ab1-5479-4d41-a5b3-e436dbbee96c from DataBase >2014-08-11 17:50:08,034 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-43) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 28435af6-8b8c-43f9-9d73-5ff28ee984c4 >2014-08-11 17:50:08,062 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 28435af6-8b8c-43f9-9d73-5ff28ee984c4), log id: 5c126ffb >2014-08-11 17:50:08,064 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=28435af6-8b8c-43f9-9d73-5ff28ee984c4), log id: 2cf7b8be >2014-08-11 17:50:08,143 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) FINISH, HSMClearTaskVDSCommand, log id: 2cf7b8be >2014-08-11 17:50:08,143 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-43) FINISH, SPMClearTaskVDSCommand, log id: 5c126ffb >2014-08-11 17:50:08,182 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-43) BaseAsyncTask::removeTaskFromDB: Removed task 28435af6-8b8c-43f9-9d73-5ff28ee984c4 from DataBase >2014-08-11 17:50:08,183 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-43) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 8caa65a6-3cf4-433a-ae78-15c6bfab9aef >2014-08-11 17:51:17,427 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-65) Setting new tasks map. The map contains now 0 tasks >2014-08-11 17:51:17,428 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-65) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 18:24:00,895 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [71cc42bb] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 18:24:00,975 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] Running command: HibernateVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 18:24:00,997 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@cdd60e42), log id: 1ea926f4 >2014-08-11 18:24:00,998 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] FINISH, UpdateVmDynamicDataVDSCommand, log id: 1ea926f4 >2014-08-11 18:24:01,010 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 7a0f05bf-0abb-488d-9cf0-c2e6da1050fd, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = 720c665a-ff99-4d8e-85b0-53b98dfe6ac0, newImageDescription = ), log id: 75e9ec35 >2014-08-11 18:24:01,011 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 18:24:01,402 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] FINISH, CreateImageVDSCommand, return: 720c665a-ff99-4d8e-85b0-53b98dfe6ac0, log id: 75e9ec35 >2014-08-11 18:24:01,414 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command ee10bce3-a0d7-40ca-ae02-b35982eb6a37 >2014-08-11 18:24:01,415 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] CommandMultiAsyncTasks::AttachTask: Attaching task 241ccd45-364a-49e4-9024-629b90506892 to command ee10bce3-a0d7-40ca-ae02-b35982eb6a37. >2014-08-11 18:24:01,473 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] Adding task 241ccd45-364a-49e4-9024-629b90506892 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 18:24:01,513 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 07fbd17f-2724-4f7f-b6d3-66529b23d98e, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = 74320463-ccbf-4380-b76d-b86eed57c19e, newImageDescription = ), log id: 1f4c7aff >2014-08-11 18:24:01,515 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 18:24:02,166 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] FINISH, CreateImageVDSCommand, return: 74320463-ccbf-4380-b76d-b86eed57c19e, log id: 1f4c7aff >2014-08-11 18:24:02,185 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] CommandMultiAsyncTasks::AttachTask: Attaching task f9993971-656d-48b4-ad5b-07043434d8d6 to command ee10bce3-a0d7-40ca-ae02-b35982eb6a37. >2014-08-11 18:24:02,242 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] Adding task f9993971-656d-48b4-ad5b-07043434d8d6 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 18:24:02,270 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@84f97a7d), log id: 1bad5e5e >2014-08-11 18:24:02,283 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] FINISH, UpdateVmDynamicDataVDSCommand, log id: 1bad5e5e >2014-08-11 18:24:02,325 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] Correlation ID: 71cc42bb, Job ID: 1b836d1f-1fd2-4737-b8c5-6446453c4af0, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm was initiated by User admin (Host: h1). >2014-08-11 18:24:02,326 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] BaseAsyncTask::startPollingTask: Starting to poll task 241ccd45-364a-49e4-9024-629b90506892. >2014-08-11 18:24:02,326 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] BaseAsyncTask::startPollingTask: Starting to poll task f9993971-656d-48b4-ad5b-07043434d8d6. >2014-08-11 18:24:02,327 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-38) [71cc42bb] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 18:24:07,946 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-81) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 18:24:08,011 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) SPMAsyncTask::PollTask: Polling task 241ccd45-364a-49e4-9024-629b90506892 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 18:24:08,023 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) BaseAsyncTask::onTaskEndSuccess: Task 241ccd45-364a-49e4-9024-629b90506892 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 18:24:08,024 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-81) Task with DB Task ID a554b68b-8b10-487f-a666-184cec548142 and VDSM Task ID f9993971-656d-48b4-ad5b-07043434d8d6 is in state Polling. End action for command ee10bce3-a0d7-40ca-ae02-b35982eb6a37 will proceed when all the entitys tasks are completed. >2014-08-11 18:24:08,024 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) SPMAsyncTask::PollTask: Polling task f9993971-656d-48b4-ad5b-07043434d8d6 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 18:24:08,031 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-81) BaseAsyncTask::onTaskEndSuccess: Task f9993971-656d-48b4-ad5b-07043434d8d6 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 18:24:08,032 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-81) CommandAsyncTask::endActionIfNecessary: All tasks of command ee10bce3-a0d7-40ca-ae02-b35982eb6a37 has ended -> executing endAction >2014-08-11 18:24:08,032 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-81) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: ee10bce3-a0d7-40ca-ae02-b35982eb6a37): calling endAction . >2014-08-11 18:24:08,033 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-39) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-11 18:24:08,089 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-39) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-11 18:24:08,100 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-39) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,7a0f05bf-0abb-488d-9cf0-c2e6da1050fd,720c665a-ff99-4d8e-85b0-53b98dfe6ac0,07fbd17f-2724-4f7f-b6d3-66529b23d98e,74320463-ccbf-4380-b76d-b86eed57c19e), log id: 4e0fdb4f >2014-08-11 18:24:08,102 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-39) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,7a0f05bf-0abb-488d-9cf0-c2e6da1050fd,720c665a-ff99-4d8e-85b0-53b98dfe6ac0,07fbd17f-2724-4f7f-b6d3-66529b23d98e,74320463-ccbf-4380-b76d-b86eed57c19e), log id: 1988959c >2014-08-11 18:24:08,116 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-39) FINISH, HibernateBrokerVDSCommand, log id: 1988959c >2014-08-11 18:24:08,122 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-39) FINISH, HibernateVDSCommand, log id: 4e0fdb4f >2014-08-11 18:24:08,123 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-39) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-11 18:24:08,124 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-39) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-11 18:24:08,131 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-39) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 241ccd45-364a-49e4-9024-629b90506892 >2014-08-11 18:24:08,131 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-39) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 241ccd45-364a-49e4-9024-629b90506892), log id: 5de6e50 >2014-08-11 18:24:08,133 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-39) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=241ccd45-364a-49e4-9024-629b90506892), log id: 4e1c4932 >2014-08-11 18:24:08,242 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-39) FINISH, HSMClearTaskVDSCommand, log id: 4e1c4932 >2014-08-11 18:24:08,242 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-39) FINISH, SPMClearTaskVDSCommand, log id: 5de6e50 >2014-08-11 18:24:08,255 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-39) BaseAsyncTask::removeTaskFromDB: Removed task 241ccd45-364a-49e4-9024-629b90506892 from DataBase >2014-08-11 18:24:08,255 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-39) SPMAsyncTask::ClearAsyncTask: Attempting to clear task f9993971-656d-48b4-ad5b-07043434d8d6 >2014-08-11 18:24:08,256 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-39) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = f9993971-656d-48b4-ad5b-07043434d8d6), log id: 40afc3d4 >2014-08-11 18:24:08,257 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-39) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=f9993971-656d-48b4-ad5b-07043434d8d6), log id: 5361b479 >2014-08-11 18:24:08,331 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-39) FINISH, HSMClearTaskVDSCommand, log id: 5361b479 >2014-08-11 18:24:08,331 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-39) FINISH, SPMClearTaskVDSCommand, log id: 40afc3d4 >2014-08-11 18:24:08,338 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-39) BaseAsyncTask::removeTaskFromDB: Removed task f9993971-656d-48b4-ad5b-07043434d8d6 from DataBase >2014-08-11 18:24:08,338 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-39) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity ee10bce3-a0d7-40ca-ae02-b35982eb6a37 >2014-08-11 18:24:10,426 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-73) START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, force=false, secondsToWait=0, gracefully=false, reason=), log id: 3dfb571d >2014-08-11 18:24:10,436 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, DestroyVDSCommand, log id: 3dfb571d >2014-08-11 18:24:10,509 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-73) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm on Host h1 is suspended. >2014-08-11 18:24:10,511 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) VM vm (14c0fa91-1814-438b-8d7a-4e7de1e1da25) is running in db and not running in VDS h1 >2014-08-11 18:24:10,511 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-73) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 78d2e20e >2014-08-11 18:24:10,515 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-73) FINISH, FullListVdsCommand, return: [], log id: 78d2e20e >2014-08-11 18:24:10,545 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-50) [973d34] Running command: ProcessDownVmCommand internal: true. >2014-08-11 18:25:17,447 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-5) Setting new tasks map. The map contains now 0 tasks >2014-08-11 18:25:17,447 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-5) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 18:28:38,736 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-96) Principal admin::internal synchronized >2014-08-11 18:28:43,623 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-95) Attempting to update VMs/Templates Ovf. >2014-08-11 18:37:41,411 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [79c9a71e] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 18:37:41,474 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [79c9a71e] START, IsVmDuringInitiatingVDSCommand( vmId = 14c0fa91-1814-438b-8d7a-4e7de1e1da25), log id: 6c1b4abe >2014-08-11 18:37:41,475 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [79c9a71e] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 6c1b4abe >2014-08-11 18:37:41,533 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-31) [79c9a71e] Running command: RunVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 18:37:41,566 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-31) [79c9a71e] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@b262da76), log id: 1e04164f >2014-08-11 18:37:41,575 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-31) [79c9a71e] FINISH, UpdateVmDynamicDataVDSCommand, log id: 1e04164f >2014-08-11 18:37:41,580 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-31) [79c9a71e] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, vm=VM [vm]), log id: 4e166c18 >2014-08-11 18:37:41,584 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-31) [79c9a71e] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, vm=VM [vm]), log id: 2d80cc93 >2014-08-11 18:37:41,596 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-31) [79c9a71e] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,hiberVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,7a0f05bf-0abb-488d-9cf0-c2e6da1050fd,720c665a-ff99-4d8e-85b0-53b98dfe6ac0,07fbd17f-2724-4f7f-b6d3-66529b23d98e,74320463-ccbf-4380-b76d-b86eed57c19e,transparentHugePages=true,vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=68e21ff2-4b5a-4b29-8765-eec79f1b15dd}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=496deb19-46c7-4b58-94a3-06b7fa34f7ac}, {shared=false, index=0, volumeID=3f8b5bbe-741b-4a4f-9397-84123d530772, propagateErrors=off, format=raw, type=disk, iface=virtio, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, imageID=99e5cda5-262b-4469-b336-04e3f0cf7c38, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=99e5cda5-262b-4469-b336-04e3f0cf7c38}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:57, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=1e5b18b9-16eb-4b96-83b6-4568b8bed355}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=3959560b-27d1-4acf-acd9-c39f0c4148b2}],vmName=vm,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 18:37:41,681 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-31) [79c9a71e] FINISH, CreateVDSCommand, log id: 2d80cc93 >2014-08-11 18:37:41,699 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-31) [79c9a71e] FINISH, CreateVmVDSCommand, return: RestoringState, log id: 4e166c18 >2014-08-11 18:37:41,700 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-31) [79c9a71e] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 18:37:41,741 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-31) [79c9a71e] Correlation ID: 79c9a71e, Job ID: 0dafea3e-f1bb-44cd-bd82-8c4df31a11fa, Call Stack: null, Custom Event ID: -1, Message: VM vm was started by admin (Host: h1). >2014-08-11 18:37:43,647 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) VM vm 14c0fa91-1814-438b-8d7a-4e7de1e1da25 moved from RestoringState --> Up >2014-08-11 18:37:43,648 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-41) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 2a9f93f7 >2014-08-11 18:37:43,657 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-41) FINISH, FullListVdsCommand, return: [{displaySecurePort=5902, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=26486, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@52ab52fd, vmName=vm, fileTransferEnable=true, cpuType=SandyBridge}], log id: 2a9f93f7 >2014-08-11 18:37:43,660 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) Received a memballoon Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 18:37:43,661 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) Received a spice Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5902, type=graphics} >2014-08-11 18:37:43,779 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) Correlation ID: 79c9a71e, Job ID: 0dafea3e-f1bb-44cd-bd82-8c4df31a11fa, Call Stack: null, Custom Event ID: -1, Message: VM vm started on Host h1 >2014-08-11 18:37:43,788 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (DefaultQuartzScheduler_Worker-41) [3322697f] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-11 18:37:43,837 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-41) [3322697f] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 7a0f05bf-0abb-488d-9cf0-c2e6da1050fd, postZeros = false, forceDelete = false), log id: 6ed542f7 >2014-08-11 18:37:44,643 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-41) [3322697f] FINISH, DeleteImageGroupVDSCommand, log id: 6ed542f7 >2014-08-11 18:37:44,682 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [3322697f] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command d663fdf1-e2b9-4512-8bda-19f231efd73b >2014-08-11 18:37:44,683 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-41) [3322697f] CommandMultiAsyncTasks::AttachTask: Attaching task 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 to command d663fdf1-e2b9-4512-8bda-19f231efd73b. >2014-08-11 18:37:44,698 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-41) [3322697f] Adding task 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 18:37:44,723 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-41) [3322697f] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 07fbd17f-2724-4f7f-b6d3-66529b23d98e, postZeros = false, forceDelete = false), log id: 7044e145 >2014-08-11 18:37:46,427 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-41) [3322697f] FINISH, DeleteImageGroupVDSCommand, log id: 7044e145 >2014-08-11 18:37:46,454 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-41) [3322697f] CommandMultiAsyncTasks::AttachTask: Attaching task 4db99cdd-a845-48fc-b97d-e8fda3889250 to command d663fdf1-e2b9-4512-8bda-19f231efd73b. >2014-08-11 18:37:46,470 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-41) [3322697f] Adding task 4db99cdd-a845-48fc-b97d-e8fda3889250 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 18:37:46,503 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [3322697f] BaseAsyncTask::startPollingTask: Starting to poll task 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7. >2014-08-11 18:37:46,504 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [3322697f] BaseAsyncTask::startPollingTask: Starting to poll task 4db99cdd-a845-48fc-b97d-e8fda3889250. >2014-08-11 18:37:46,505 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-41) [3322697f] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@39c4fe0f), log id: 2d818351 >2014-08-11 18:37:46,511 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-41) [3322697f] FINISH, UpdateVmDynamicDataVDSCommand, log id: 2d818351 >2014-08-11 18:37:46,754 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [56c3ad5f] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 18:37:46,755 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [56c3ad5f] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:255) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 18:37:46,788 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [56c3ad5f] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-11 18:37:48,049 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-62) task id 4db99cdd-a845-48fc-b97d-e8fda3889250 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:37:48,050 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-62) task id 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:37:58,051 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-96) task id 4db99cdd-a845-48fc-b97d-e8fda3889250 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:37:58,053 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-96) task id 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:37:58,569 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-99) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 75577e21 >2014-08-11 18:37:58,580 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-99) FINISH, FullListVdsCommand, return: [{displaySecurePort=5902, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=26486, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@4319e4b6, vmName=vm, fileTransferEnable=true, cpuType=SandyBridge}], log id: 75577e21 >2014-08-11 18:37:58,583 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Received a memballoon Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 18:37:58,584 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Received a spice Device without an address when processing VM 14c0fa91-1814-438b-8d7a-4e7de1e1da25 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5902, type=graphics} >2014-08-11 18:38:08,055 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-77) task id 4db99cdd-a845-48fc-b97d-e8fda3889250 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:08,055 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-77) task id 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:18,057 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-24) task id 4db99cdd-a845-48fc-b97d-e8fda3889250 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:18,057 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-24) task id 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:28,057 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-39) task id 4db99cdd-a845-48fc-b97d-e8fda3889250 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:28,059 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-39) task id 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:38,060 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-57) task id 4db99cdd-a845-48fc-b97d-e8fda3889250 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:38,062 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-57) task id 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:48,063 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) task id 4db99cdd-a845-48fc-b97d-e8fda3889250 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:48,065 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) task id 4db99cdd-a845-48fc-b97d-e8fda3889250 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:48,066 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) task id 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:48,067 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-98) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 18:38:48,067 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) task id 4db99cdd-a845-48fc-b97d-e8fda3889250 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:48,068 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) task id 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:48,074 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) task id 4db99cdd-a845-48fc-b97d-e8fda3889250 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:48,075 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) SPMAsyncTask::PollTask: Polling task 4db99cdd-a845-48fc-b97d-e8fda3889250 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 18:38:48,149 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) BaseAsyncTask::onTaskEndSuccess: Task 4db99cdd-a845-48fc-b97d-e8fda3889250 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 18:38:48,149 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-98) Task with DB Task ID ebf082ba-951e-4a0a-a897-fc7e70fd4d17 and VDSM Task ID 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 is in state Polling. End action for command d663fdf1-e2b9-4512-8bda-19f231efd73b will proceed when all the entitys tasks are completed. >2014-08-11 18:38:48,150 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) task id 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:38:48,151 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) SPMAsyncTask::PollTask: Polling task 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 18:38:48,165 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) BaseAsyncTask::onTaskEndSuccess: Task 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 18:38:48,166 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-98) CommandAsyncTask::endActionIfNecessary: All tasks of command d663fdf1-e2b9-4512-8bda-19f231efd73b has ended -> executing endAction >2014-08-11 18:38:48,166 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-98) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: d663fdf1-e2b9-4512-8bda-19f231efd73b): calling endAction . >2014-08-11 18:38:48,167 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-48) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RunVm, executionIndex: 0 >2014-08-11 18:38:48,182 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-48) Ending command successfully: org.ovirt.engine.core.bll.RunVmCommand >2014-08-11 18:38:48,204 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm completed, handling the result. >2014-08-11 18:38:48,204 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm succeeded, clearing tasks. >2014-08-11 18:38:48,215 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-48) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 >2014-08-11 18:38:48,215 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7), log id: 1c0adb76 >2014-08-11 18:38:48,217 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=9f0a4988-56d3-45eb-baa0-fdd53e1de7c7), log id: cce2e5c >2014-08-11 18:38:48,288 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, HSMClearTaskVDSCommand, log id: cce2e5c >2014-08-11 18:38:48,288 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, SPMClearTaskVDSCommand, log id: 1c0adb76 >2014-08-11 18:38:48,298 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-48) BaseAsyncTask::removeTaskFromDB: Removed task 9f0a4988-56d3-45eb-baa0-fdd53e1de7c7 from DataBase >2014-08-11 18:38:48,298 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-48) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 4db99cdd-a845-48fc-b97d-e8fda3889250 >2014-08-11 18:38:48,299 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 4db99cdd-a845-48fc-b97d-e8fda3889250), log id: 7a13e05c >2014-08-11 18:38:48,300 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=4db99cdd-a845-48fc-b97d-e8fda3889250), log id: 2eeef6a1 >2014-08-11 18:38:48,362 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, HSMClearTaskVDSCommand, log id: 2eeef6a1 >2014-08-11 18:38:48,363 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, SPMClearTaskVDSCommand, log id: 7a13e05c >2014-08-11 18:38:48,372 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-48) BaseAsyncTask::removeTaskFromDB: Removed task 4db99cdd-a845-48fc-b97d-e8fda3889250 from DataBase >2014-08-11 18:38:48,373 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity d663fdf1-e2b9-4512-8bda-19f231efd73b >2014-08-11 18:40:17,456 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-26) Setting new tasks map. The map contains now 0 tasks >2014-08-11 18:40:17,457 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-26) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 18:54:41,274 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (http--0.0.0.0-8080-2) [dc6f829] Lock Acquired to object EngineLock [exclusiveLocks= key: vm_rhel_7 value: VM_NAME >, sharedLocks= ] >2014-08-11 18:54:41,337 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (http--0.0.0.0-8080-2) [dc6f829] Running command: AddVmFromScratchCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-00000000039f Type: VdsGroupsAction group CREATE_VM with role type USER >2014-08-11 18:54:41,544 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [dc6f829] Correlation ID: dc6f829, Job ID: 3dd8dc0d-4f5a-4189-8162-d5f5683317d5, Call Stack: null, Custom Event ID: -1, Message: VM vm_rhel_7 was created by admin. >2014-08-11 18:54:41,553 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (http--0.0.0.0-8080-2) [dc6f829] Lock freed to object EngineLock [exclusiveLocks= key: vm_rhel_7 value: VM_NAME >, sharedLocks= ] >2014-08-11 18:54:41,996 INFO [org.ovirt.engine.core.bll.network.vm.AddVmInterfaceCommand] (http--0.0.0.0-8080-2) [2769a0f7] Running command: AddVmInterfaceCommand internal: false. Entities affected : ID: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 Type: VMAction group CONFIGURE_VM_NETWORK with role type USER, ID: 0000000e-000e-000e-000e-000000000181 Type: VnicProfileAction group CONFIGURE_VM_NETWORK with role type USER >2014-08-11 18:54:42,042 INFO [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (http--0.0.0.0-8080-2) [3969f181] Running command: ActivateDeactivateVmNicCommand internal: true. Entities affected : ID: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 Type: VM >2014-08-11 18:54:42,074 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [3969f181] Correlation ID: 3969f181, Call Stack: null, Custom Event ID: -1, Message: Network Interface nic1 (VirtIO) was plugged to VM vm_rhel_7. (User: admin) >2014-08-11 18:54:42,099 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [3969f181] Correlation ID: 2769a0f7, Call Stack: null, Custom Event ID: -1, Message: Interface nic1 (VirtIO) was added to VM vm_rhel_7. (User: admin) >2014-08-11 18:54:42,119 INFO [org.ovirt.engine.core.bll.network.vm.ReorderVmNicsCommand] (http--0.0.0.0-8080-2) [72aa49f7] Running command: ReorderVmNicsCommand internal: false. Entities affected : ID: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 Type: VMAction group CONFIGURE_VM_NETWORK with role type USER >2014-08-11 18:54:49,469 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (http--0.0.0.0-8080-2) [48494005] Lock Acquired to object EngineLock [exclusiveLocks= key: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 value: VM_DISK_BOOT >, sharedLocks= key: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 value: VM >] >2014-08-11 18:54:49,495 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (http--0.0.0.0-8080-2) [48494005] Running command: AddDiskCommand internal: false. Entities affected : ID: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 Type: VMAction group CONFIGURE_VM_STORAGE with role type USER, ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: StorageAction group CREATE_DISK with role type USER >2014-08-11 18:54:49,570 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (http--0.0.0.0-8080-2) [1434b01a] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-11 18:54:49,587 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (http--0.0.0.0-8080-2) [1434b01a] Lock freed to object EngineLock [exclusiveLocks= key: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 value: VM_DISK_BOOT >, sharedLocks= key: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 value: VM >] >2014-08-11 18:54:49,587 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (http--0.0.0.0-8080-2) [1434b01a] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 9bbaea85-87de-4005-9f80-7dbc97cff94a, imageSizeInBytes = 10737418240, volumeFormat = RAW, newImageId = 1c3f60ad-bf75-4e71-b119-1d00c1718087, newImageDescription = ), log id: 3fa77c7 >2014-08-11 18:54:49,589 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (http--0.0.0.0-8080-2) [1434b01a] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 18:54:49,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (http--0.0.0.0-8080-2) [1434b01a] FINISH, CreateImageVDSCommand, return: 1c3f60ad-bf75-4e71-b119-1d00c1718087, log id: 3fa77c7 >2014-08-11 18:54:49,996 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (http--0.0.0.0-8080-2) [1434b01a] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 1cdf064a-a0d8-4294-8966-7b0ec9b01474 >2014-08-11 18:54:49,996 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (http--0.0.0.0-8080-2) [1434b01a] CommandMultiAsyncTasks::AttachTask: Attaching task 905c1e6e-58ee-431a-8e97-1972badace8c to command 1cdf064a-a0d8-4294-8966-7b0ec9b01474. >2014-08-11 18:54:50,025 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (http--0.0.0.0-8080-2) [1434b01a] Adding task 905c1e6e-58ee-431a-8e97-1972badace8c (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 18:54:50,158 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [1434b01a] Correlation ID: 48494005, Job ID: db2275cd-8ee6-4c0d-8b11-e04793a320a0, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of vm_rhel_7_Disk1 was initiated on VM vm_rhel_7 by admin. >2014-08-11 18:54:50,159 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (http--0.0.0.0-8080-2) [1434b01a] BaseAsyncTask::startPollingTask: Starting to poll task 905c1e6e-58ee-431a-8e97-1972badace8c. >2014-08-11 18:54:58,191 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-3) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-11 18:54:58,199 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) SPMAsyncTask::PollTask: Polling task 905c1e6e-58ee-431a-8e97-1972badace8c (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 18:54:58,251 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) BaseAsyncTask::onTaskEndSuccess: Task 905c1e6e-58ee-431a-8e97-1972badace8c (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 18:54:58,251 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-3) CommandAsyncTask::endActionIfNecessary: All tasks of command 1cdf064a-a0d8-4294-8966-7b0ec9b01474 has ended -> executing endAction >2014-08-11 18:54:58,252 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-3) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 1cdf064a-a0d8-4294-8966-7b0ec9b01474): calling endAction . >2014-08-11 18:54:58,252 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-27) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddDisk, executionIndex: 0 >2014-08-11 18:54:58,267 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (org.ovirt.thread.pool-8-thread-27) [48494005] Ending command successfully: org.ovirt.engine.core.bll.AddDiskCommand >2014-08-11 18:54:58,284 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-8-thread-27) [1434b01a] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-08-11 18:54:58,289 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-27) [1434b01a] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 9bbaea85-87de-4005-9f80-7dbc97cff94a, imageId = 1c3f60ad-bf75-4e71-b119-1d00c1718087), log id: 8b69d98 >2014-08-11 18:54:58,346 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-27) [1434b01a] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@5a15d136, log id: 8b69d98 >2014-08-11 18:54:58,375 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-27) Correlation ID: 48494005, Call Stack: null, Custom Event ID: -1, Message: The disk vm_rhel_7_Disk1 was successfully added to VM vm_rhel_7. >2014-08-11 18:54:58,376 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-27) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk completed, handling the result. >2014-08-11 18:54:58,376 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-27) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk succeeded, clearing tasks. >2014-08-11 18:54:58,383 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-27) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 905c1e6e-58ee-431a-8e97-1972badace8c >2014-08-11 18:54:58,384 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 905c1e6e-58ee-431a-8e97-1972badace8c), log id: 151e0dad >2014-08-11 18:54:58,385 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=905c1e6e-58ee-431a-8e97-1972badace8c), log id: 6813e3af >2014-08-11 18:54:58,481 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) FINISH, HSMClearTaskVDSCommand, log id: 6813e3af >2014-08-11 18:54:58,482 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) FINISH, SPMClearTaskVDSCommand, log id: 151e0dad >2014-08-11 18:54:58,491 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-27) BaseAsyncTask::removeTaskFromDB: Removed task 905c1e6e-58ee-431a-8e97-1972badace8c from DataBase >2014-08-11 18:54:58,491 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-27) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 1cdf064a-a0d8-4294-8966-7b0ec9b01474 >2014-08-11 18:55:00,204 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [26acfb2f] Lock Acquired to object EngineLock [exclusiveLocks= key: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 value: VM >, sharedLocks= ] >2014-08-11 18:55:00,223 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [26acfb2f] START, IsVmDuringInitiatingVDSCommand( vmId = 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4), log id: 2bf03aa1 >2014-08-11 18:55:00,223 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [26acfb2f] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 2bf03aa1 >2014-08-11 18:55:00,338 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-8) [26acfb2f] Running command: RunVmCommand internal: false. Entities affected : ID: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 18:55:00,402 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-8) [26acfb2f] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@5eeadc60), log id: 19743ff1 >2014-08-11 18:55:00,429 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-8) [26acfb2f] FINISH, UpdateVmDynamicDataVDSCommand, log id: 19743ff1 >2014-08-11 18:55:00,433 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-8) [26acfb2f] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, vm=VM [vm_rhel_7]), log id: 450d984e >2014-08-11 18:55:00,546 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand] (org.ovirt.thread.pool-8-thread-8) [26acfb2f] START, CreateVmFromCloudInitVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, vm=VM [vm_rhel_7]), log id: 226a84a8 >2014-08-11 18:55:00,556 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-8) [26acfb2f] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{shared=false, iface=ide, index=3, specParams={vmPayload={file={openstack/latest/meta_data.json=ewogICJsYXVuY2hfaW5kZXgiIDogIjAiLAogICJhdmFpbGFiaWxpdHlfem9uZSIgOiAibm92YSIs >CiAgIm5ldHdvcmstaW50ZXJmYWNlcyIgOiAiYXV0b1xuIiwKICAibmV0d29ya19jb25maWciIDog >ewogICAgImNvbnRlbnRfcGF0aCIgOiAiL2NvbnRlbnQvMDAwMCIsCiAgICAicGF0aCIgOiAiL2V0 >Yy9uZXR3b3JrL2ludGVyZmFjZXMiCiAgfSwKICAidXVpZCIgOiAiNDY2YTFjNWEtMGIzOC00ODU0 >LWE4ZDEtN2NlZWYzM2ZmNGYyIiwKICAibWV0YSIgOiB7CiAgICAiZXNzZW50aWFsIiA6ICJmYWxz >ZSIsCiAgICAicm9sZSIgOiAic2VydmVyIiwKICAgICJkc21vZGUiIDogImxvY2FsIgogIH0KfQ== >, openstack/latest/user_data=I2Nsb3VkLWNvbmZpZwpzc2hfcHdhdXRoOiB0cnVlCmRpc2FibGVfcm9vdDogMApvdXRwdXQ6CiAg >YWxsOiAnPj4gL3Zhci9sb2cvY2xvdWQtaW5pdC1vdXRwdXQubG9nJwpjaHBhc3N3ZDoKICBleHBp >cmU6IGZhbHNlCnJ1bmNtZDoKLSAnc2VkIC1pICcnL15kYXRhc291cmNlX2xpc3Q6IC9kJycgL2V0 >Yy9jbG91ZC9jbG91ZC5jZmc7IGVjaG8gJydkYXRhc291cmNlX2xpc3Q6CiAgWyJOb0Nsb3VkIiwg >IkNvbmZpZ0RyaXZlIl0nJyA+PiAvZXRjL2Nsb3VkL2Nsb3VkLmNmZycK >, openstack/content/0000=YXV0bwo= >}, volId=config-2}}, path=, device=cdrom, type=disk, readonly=true, deviceId=36c81441-8ae5-438a-9d7f-92b4e8ffd0ef}, {specParams={ram=65536, vram=32768, heads=1}, device=qxl, type=video, deviceId=7e76fc7e-1514-4944-bd5a-7b1418d85f2f}, {shared=false, iface=ide, index=2, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=fdf3b4e8-b82d-4913-b83b-5c8b5528d0bd}, {shared=false, volumeID=1c3f60ad-bf75-4e71-b119-1d00c1718087, index=0, iface=virtio, imageID=9bbaea85-87de-4005-9f80-7dbc97cff94a, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, specParams={}, optional=false, propagateErrors=off, device=disk, poolID=00000002-0002-0002-0002-00000000037f, format=raw, type=disk, readonly=false, deviceId=9bbaea85-87de-4005-9f80-7dbc97cff94a}, {nicModel=pv, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:59, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=fcd9032f-e30b-48f4-9bfa-94ce968f04ae}, {index=0, model=virtio-scsi, specParams={}, device=scsi, type=controller, deviceId=379f8e1b-e6f2-42e3-9fff-8157388d177f}],vmName=vm_rhel_7,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 18:55:00,578 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand] (org.ovirt.thread.pool-8-thread-8) [26acfb2f] FINISH, CreateVmFromCloudInitVDSCommand, log id: 226a84a8 >2014-08-11 18:55:00,621 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-8) [26acfb2f] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 450d984e >2014-08-11 18:55:00,622 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-8) [26acfb2f] Lock freed to object EngineLock [exclusiveLocks= key: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 value: VM >, sharedLocks= ] >2014-08-11 18:55:00,646 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-8) [26acfb2f] Correlation ID: 26acfb2f, Job ID: da0c8c6d-14b0-4a5b-b06d-619baa198cac, Call Stack: null, Custom Event ID: -1, Message: VM vm_rhel_7 was started by admin (Host: h1). >2014-08-11 18:55:03,504 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) VM vm_rhel_7 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 moved from WaitForLaunch --> PoweringUp >2014-08-11 18:55:03,505 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-29) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[7b4eb2dc-f23c-4c2a-8302-f0e1184470d4]), log id: c665dd5 >2014-08-11 18:55:03,516 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-29) FINISH, FullListVdsCommand, return: [{displaySecurePort=5904, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.35.0.105, pauseCode=NOERR, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=27960, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@149a514f, vmName=vm_rhel_7, fileTransferEnable=true, cpuType=SandyBridge}], log id: c665dd5 >2014-08-11 18:55:03,522 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) Received a memballoon Device without an address when processing VM 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-08-11 18:55:03,523 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) Received a spice Device without an address when processing VM 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5904, type=graphics} >2014-08-11 18:56:02,992 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) VM vm_rhel_7 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 moved from PoweringUp --> Up >2014-08-11 18:56:03,170 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) Correlation ID: 26acfb2f, Job ID: da0c8c6d-14b0-4a5b-b06d-619baa198cac, Call Stack: null, Custom Event ID: -1, Message: VM vm_rhel_7 started on Host h1 >2014-08-11 18:56:17,466 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-48) Setting new tasks map. The map contains now 0 tasks >2014-08-11 18:56:17,466 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-48) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 18:58:00,391 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [421bea06] Lock Acquired to object EngineLock [exclusiveLocks= key: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 value: VM >, sharedLocks= ] >2014-08-11 18:58:00,512 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-17) [421bea06] Running command: HibernateVmCommand internal: false. Entities affected : ID: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 18:58:00,519 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-17) [421bea06] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@34a38286), log id: 5e4834c >2014-08-11 18:58:00,521 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-17) [421bea06] FINISH, UpdateVmDynamicDataVDSCommand, log id: 5e4834c >2014-08-11 18:58:00,531 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-17) [421bea06] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 39d7bb99-f958-47d5-aa50-b0ee5d01b628, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = d5f17ad2-bffe-47f6-a833-0399839fe82c, newImageDescription = ), log id: 4590cd53 >2014-08-11 18:58:00,533 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-17) [421bea06] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 18:58:00,888 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-17) [421bea06] FINISH, CreateImageVDSCommand, return: d5f17ad2-bffe-47f6-a833-0399839fe82c, log id: 4590cd53 >2014-08-11 18:58:00,903 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-17) [421bea06] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command cf6a5429-761f-49b4-92c3-e512417bd83e >2014-08-11 18:58:00,904 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-17) [421bea06] CommandMultiAsyncTasks::AttachTask: Attaching task de607883-93e5-4ef0-b2a4-d5f548d8259c to command cf6a5429-761f-49b4-92c3-e512417bd83e. >2014-08-11 18:58:00,912 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-17) [421bea06] Adding task de607883-93e5-4ef0-b2a4-d5f548d8259c (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 18:58:00,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-17) [421bea06] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = e7299096-6df4-4767-8e26-df6cd80bdf6c, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = 3a81a1ee-be13-43f3-8648-cb473bf0cc34, newImageDescription = ), log id: 4784dd90 >2014-08-11 18:58:00,937 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-17) [421bea06] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 18:58:01,509 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-17) [421bea06] FINISH, CreateImageVDSCommand, return: 3a81a1ee-be13-43f3-8648-cb473bf0cc34, log id: 4784dd90 >2014-08-11 18:58:01,548 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-17) [421bea06] CommandMultiAsyncTasks::AttachTask: Attaching task 6e687e72-b61e-4ccc-a653-8a8872c02741 to command cf6a5429-761f-49b4-92c3-e512417bd83e. >2014-08-11 18:58:01,589 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-17) [421bea06] Adding task 6e687e72-b61e-4ccc-a653-8a8872c02741 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 18:58:01,598 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-17) [421bea06] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@dbb86e52), log id: 4b481f6a >2014-08-11 18:58:01,605 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-17) [421bea06] FINISH, UpdateVmDynamicDataVDSCommand, log id: 4b481f6a >2014-08-11 18:58:01,655 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-17) [421bea06] Correlation ID: 421bea06, Job ID: b0b05cde-46d4-4bb5-99eb-ba2045ef1baf, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm_rhel_7 was initiated by User admin (Host: h1). >2014-08-11 18:58:01,655 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-17) [421bea06] BaseAsyncTask::startPollingTask: Starting to poll task de607883-93e5-4ef0-b2a4-d5f548d8259c. >2014-08-11 18:58:01,655 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-17) [421bea06] BaseAsyncTask::startPollingTask: Starting to poll task 6e687e72-b61e-4ccc-a653-8a8872c02741. >2014-08-11 18:58:01,656 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-17) [421bea06] Lock freed to object EngineLock [exclusiveLocks= key: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 value: VM >, sharedLocks= ] >2014-08-11 18:58:05,013 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) [4337c859] VM vm_rhel_7 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 moved from PreparingForHibernate --> Up >2014-08-11 18:58:08,256 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-49) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 18:58:08,263 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) SPMAsyncTask::PollTask: Polling task de607883-93e5-4ef0-b2a4-d5f548d8259c (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 18:58:08,389 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) BaseAsyncTask::onTaskEndSuccess: Task de607883-93e5-4ef0-b2a4-d5f548d8259c (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 18:58:08,391 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-49) Task with DB Task ID 578a6882-931d-457f-886c-eddc884fe816 and VDSM Task ID 6e687e72-b61e-4ccc-a653-8a8872c02741 is in state Polling. End action for command cf6a5429-761f-49b4-92c3-e512417bd83e will proceed when all the entitys tasks are completed. >2014-08-11 18:58:08,392 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) SPMAsyncTask::PollTask: Polling task 6e687e72-b61e-4ccc-a653-8a8872c02741 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 18:58:08,406 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) BaseAsyncTask::onTaskEndSuccess: Task 6e687e72-b61e-4ccc-a653-8a8872c02741 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 18:58:08,407 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-49) CommandAsyncTask::endActionIfNecessary: All tasks of command cf6a5429-761f-49b4-92c3-e512417bd83e has ended -> executing endAction >2014-08-11 18:58:08,408 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-49) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: cf6a5429-761f-49b4-92c3-e512417bd83e): calling endAction . >2014-08-11 18:58:08,409 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-11 18:58:08,488 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-14) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-11 18:58:08,491 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,39d7bb99-f958-47d5-aa50-b0ee5d01b628,d5f17ad2-bffe-47f6-a833-0399839fe82c,e7299096-6df4-4767-8e26-df6cd80bdf6c,3a81a1ee-be13-43f3-8648-cb473bf0cc34), log id: 5623f990 >2014-08-11 18:58:08,493 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,39d7bb99-f958-47d5-aa50-b0ee5d01b628,d5f17ad2-bffe-47f6-a833-0399839fe82c,e7299096-6df4-4767-8e26-df6cd80bdf6c,3a81a1ee-be13-43f3-8648-cb473bf0cc34), log id: bebd5c4 >2014-08-11 18:58:08,501 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, HibernateBrokerVDSCommand, log id: bebd5c4 >2014-08-11 18:58:08,513 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, HibernateVDSCommand, log id: 5623f990 >2014-08-11 18:58:08,515 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-11 18:58:08,515 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-11 18:58:08,521 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) SPMAsyncTask::ClearAsyncTask: Attempting to clear task de607883-93e5-4ef0-b2a4-d5f548d8259c >2014-08-11 18:58:08,522 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = de607883-93e5-4ef0-b2a4-d5f548d8259c), log id: 5cdb0621 >2014-08-11 18:58:08,524 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=de607883-93e5-4ef0-b2a4-d5f548d8259c), log id: 2f6f9027 >2014-08-11 18:58:08,624 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, HSMClearTaskVDSCommand, log id: 2f6f9027 >2014-08-11 18:58:08,625 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, SPMClearTaskVDSCommand, log id: 5cdb0621 >2014-08-11 18:58:08,637 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) BaseAsyncTask::removeTaskFromDB: Removed task de607883-93e5-4ef0-b2a4-d5f548d8259c from DataBase >2014-08-11 18:58:08,638 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 6e687e72-b61e-4ccc-a653-8a8872c02741 >2014-08-11 18:58:08,639 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 6e687e72-b61e-4ccc-a653-8a8872c02741), log id: bac4e3e >2014-08-11 18:58:08,642 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=6e687e72-b61e-4ccc-a653-8a8872c02741), log id: 5d44ce6e >2014-08-11 18:58:08,720 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, HSMClearTaskVDSCommand, log id: 5d44ce6e >2014-08-11 18:58:08,721 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, SPMClearTaskVDSCommand, log id: bac4e3e >2014-08-11 18:58:08,728 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) BaseAsyncTask::removeTaskFromDB: Removed task 6e687e72-b61e-4ccc-a653-8a8872c02741 from DataBase >2014-08-11 18:58:08,729 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity cf6a5429-761f-49b4-92c3-e512417bd83e >2014-08-11 18:58:11,155 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-47) START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, force=false, secondsToWait=0, gracefully=false, reason=), log id: 4ead82a >2014-08-11 18:58:11,165 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, DestroyVDSCommand, log id: 4ead82a >2014-08-11 18:58:11,254 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm_rhel_7 on Host h1 is suspended. >2014-08-11 18:58:11,256 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) VM vm_rhel_7 (7b4eb2dc-f23c-4c2a-8302-f0e1184470d4) is running in db and not running in VDS h1 >2014-08-11 18:58:11,256 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-47) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[7b4eb2dc-f23c-4c2a-8302-f0e1184470d4]), log id: 6d58fdf3 >2014-08-11 18:58:11,260 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-47) FINISH, FullListVdsCommand, return: [], log id: 6d58fdf3 >2014-08-11 18:58:11,290 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-18) [1e259a4a] Running command: ProcessDownVmCommand internal: true. >2014-08-11 18:58:36,644 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [368e65ac] Lock Acquired to object EngineLock [exclusiveLocks= key: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 value: VM >, sharedLocks= ] >2014-08-11 18:58:36,668 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [368e65ac] START, IsVmDuringInitiatingVDSCommand( vmId = 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4), log id: 5a212188 >2014-08-11 18:58:36,668 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [368e65ac] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 5a212188 >2014-08-11 18:58:36,758 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-24) [368e65ac] Running command: RunVmCommand internal: false. Entities affected : ID: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 18:58:36,787 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-24) [368e65ac] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@f8e7d7c4), log id: 78578fc1 >2014-08-11 18:58:36,799 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-24) [368e65ac] FINISH, UpdateVmDynamicDataVDSCommand, log id: 78578fc1 >2014-08-11 18:58:36,802 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-24) [368e65ac] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, vm=VM [vm_rhel_7]), log id: 10771afb >2014-08-11 18:58:36,805 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-24) [368e65ac] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, vm=VM [vm_rhel_7]), log id: 556eddbc >2014-08-11 18:58:36,815 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-24) [368e65ac] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,hiberVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,39d7bb99-f958-47d5-aa50-b0ee5d01b628,d5f17ad2-bffe-47f6-a833-0399839fe82c,e7299096-6df4-4767-8e26-df6cd80bdf6c,3a81a1ee-be13-43f3-8648-cb473bf0cc34,transparentHugePages=true,vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={ram=65536, vram=32768, heads=1}, device=qxl, type=video, deviceId=7e76fc7e-1514-4944-bd5a-7b1418d85f2f}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=fdf3b4e8-b82d-4913-b83b-5c8b5528d0bd}, {shared=false, index=0, volumeID=1c3f60ad-bf75-4e71-b119-1d00c1718087, propagateErrors=off, format=raw, type=disk, iface=virtio, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, imageID=9bbaea85-87de-4005-9f80-7dbc97cff94a, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=9bbaea85-87de-4005-9f80-7dbc97cff94a}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:59, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=fcd9032f-e30b-48f4-9bfa-94ce968f04ae}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=379f8e1b-e6f2-42e3-9fff-8157388d177f}],vmName=vm_rhel_7,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 18:58:36,878 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-24) [368e65ac] FINISH, CreateVDSCommand, log id: 556eddbc >2014-08-11 18:58:36,898 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-24) [368e65ac] FINISH, CreateVmVDSCommand, return: RestoringState, log id: 10771afb >2014-08-11 18:58:36,899 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-24) [368e65ac] Lock freed to object EngineLock [exclusiveLocks= key: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 value: VM >, sharedLocks= ] >2014-08-11 18:58:36,956 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-24) [368e65ac] Correlation ID: 368e65ac, Job ID: b867b099-d473-4479-9f40-2c9caf65c34d, Call Stack: null, Custom Event ID: -1, Message: VM vm_rhel_7 was started by admin (Host: h1). >2014-08-11 18:58:38,784 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) VM vm_rhel_7 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 moved from RestoringState --> Up >2014-08-11 18:58:38,784 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-88) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[7b4eb2dc-f23c-4c2a-8302-f0e1184470d4]), log id: 77187cdb >2014-08-11 18:58:38,794 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-88) FINISH, FullListVdsCommand, return: [{displaySecurePort=5904, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=28440, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@68ad4af2, vmName=vm_rhel_7, fileTransferEnable=true, cpuType=SandyBridge}], log id: 77187cdb >2014-08-11 18:58:38,797 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Received a memballoon Device without an address when processing VM 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 18:58:38,798 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Received a spice Device without an address when processing VM 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5904, type=graphics} >2014-08-11 18:58:38,870 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-88) Correlation ID: 368e65ac, Job ID: b867b099-d473-4479-9f40-2c9caf65c34d, Call Stack: null, Custom Event ID: -1, Message: VM vm_rhel_7 started on Host h1 >2014-08-11 18:58:38,879 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (DefaultQuartzScheduler_Worker-88) [5d538ea9] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-11 18:58:38,911 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-88) [5d538ea9] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 39d7bb99-f958-47d5-aa50-b0ee5d01b628, postZeros = false, forceDelete = false), log id: 7f973cab >2014-08-11 18:58:39,349 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-88) [5d538ea9] FINISH, DeleteImageGroupVDSCommand, log id: 7f973cab >2014-08-11 18:58:39,359 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-88) [5d538ea9] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command faae2f3e-ca48-49dd-9adc-aa5e63317ed4 >2014-08-11 18:58:39,360 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-88) [5d538ea9] CommandMultiAsyncTasks::AttachTask: Attaching task f8414158-0ec0-4852-80cd-1f97eabe42eb to command faae2f3e-ca48-49dd-9adc-aa5e63317ed4. >2014-08-11 18:58:39,375 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-88) [5d538ea9] Adding task f8414158-0ec0-4852-80cd-1f97eabe42eb (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 18:58:39,400 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-88) [5d538ea9] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = e7299096-6df4-4767-8e26-df6cd80bdf6c, postZeros = false, forceDelete = false), log id: 91521ff >2014-08-11 18:58:39,966 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-88) [5d538ea9] FINISH, DeleteImageGroupVDSCommand, log id: 91521ff >2014-08-11 18:58:40,022 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-88) [5d538ea9] CommandMultiAsyncTasks::AttachTask: Attaching task d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 to command faae2f3e-ca48-49dd-9adc-aa5e63317ed4. >2014-08-11 18:58:40,037 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-88) [5d538ea9] Adding task d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 18:58:40,104 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) [5d538ea9] BaseAsyncTask::startPollingTask: Starting to poll task f8414158-0ec0-4852-80cd-1f97eabe42eb. >2014-08-11 18:58:40,104 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) [5d538ea9] BaseAsyncTask::startPollingTask: Starting to poll task d55803cf-0c5c-460d-a5cf-8c7705a3ebf0. >2014-08-11 18:58:40,106 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-88) [5d538ea9] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@a2c15da0), log id: 1b7d25bc >2014-08-11 18:58:40,120 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-88) [5d538ea9] FINISH, UpdateVmDynamicDataVDSCommand, log id: 1b7d25bc >2014-08-11 18:58:48,189 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [72bc8b42] Lock Acquired to object EngineLock [exclusiveLocks= key: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 value: VM >, sharedLocks= ] >2014-08-11 18:58:48,190 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [72bc8b42] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:255) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 18:58:48,226 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [72bc8b42] Lock freed to object EngineLock [exclusiveLocks= key: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 value: VM >, sharedLocks= ] >2014-08-11 18:58:48,409 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-30) task id d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:58:48,410 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-30) task id f8414158-0ec0-4852-80cd-1f97eabe42eb is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:58:52,222 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-36) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[7b4eb2dc-f23c-4c2a-8302-f0e1184470d4]), log id: 493b390d >2014-08-11 18:58:52,231 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-36) FINISH, FullListVdsCommand, return: [{displaySecurePort=5904, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, acpiEnable=true, custom={}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=28440, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@4f1eaa0a, vmName=vm_rhel_7, fileTransferEnable=true, cpuType=SandyBridge}], log id: 493b390d >2014-08-11 18:58:52,233 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Received a memballoon Device without an address when processing VM 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 18:58:52,234 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Received a spice Device without an address when processing VM 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5904, type=graphics} >2014-08-11 18:58:58,411 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-32) [4337c859] task id d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:58:58,412 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-32) [4337c859] task id f8414158-0ec0-4852-80cd-1f97eabe42eb is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:08,413 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-65) task id d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:08,414 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-65) task id f8414158-0ec0-4852-80cd-1f97eabe42eb is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:17,468 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-83) Setting new tasks map. The map contains now 2 tasks >2014-08-11 18:59:18,414 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) task id d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:18,416 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-98) task id f8414158-0ec0-4852-80cd-1f97eabe42eb is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:28,363 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (http--0.0.0.0-8080-2) [7a495694] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 Type: VMAction group CONNECT_TO_VM with role type USER >2014-08-11 18:59:28,365 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-2) [7a495694] START, SetVmTicketVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, ticket=k6ln3Nd0dAQE, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 685d18c4 >2014-08-11 18:59:28,379 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-2) [7a495694] FINISH, SetVmTicketVDSCommand, log id: 685d18c4 >2014-08-11 18:59:28,417 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-97) task id d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:28,418 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-97) task id f8414158-0ec0-4852-80cd-1f97eabe42eb is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:28,452 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [7a495694] Correlation ID: 7a495694, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM vm_rhel_7 >2014-08-11 18:59:38,420 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-5) task id d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:38,421 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-5) task id f8414158-0ec0-4852-80cd-1f97eabe42eb is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:48,422 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) task id d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:48,423 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) task id d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:48,424 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) task id f8414158-0ec0-4852-80cd-1f97eabe42eb has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:48,425 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-21) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 18:59:48,426 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) task id d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:48,427 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) task id f8414158-0ec0-4852-80cd-1f97eabe42eb has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:48,433 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) task id d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:48,433 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) SPMAsyncTask::PollTask: Polling task d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 18:59:48,472 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) BaseAsyncTask::onTaskEndSuccess: Task d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 18:59:48,473 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-21) Task with DB Task ID 8c373e08-f181-417b-9572-44fea6d1ca5f and VDSM Task ID f8414158-0ec0-4852-80cd-1f97eabe42eb is in state Polling. End action for command faae2f3e-ca48-49dd-9adc-aa5e63317ed4 will proceed when all the entitys tasks are completed. >2014-08-11 18:59:48,473 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) task id f8414158-0ec0-4852-80cd-1f97eabe42eb has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 18:59:48,474 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) SPMAsyncTask::PollTask: Polling task f8414158-0ec0-4852-80cd-1f97eabe42eb (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 18:59:48,488 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-21) BaseAsyncTask::onTaskEndSuccess: Task f8414158-0ec0-4852-80cd-1f97eabe42eb (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 18:59:48,489 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-21) CommandAsyncTask::endActionIfNecessary: All tasks of command faae2f3e-ca48-49dd-9adc-aa5e63317ed4 has ended -> executing endAction >2014-08-11 18:59:48,489 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-21) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: faae2f3e-ca48-49dd-9adc-aa5e63317ed4): calling endAction . >2014-08-11 18:59:48,490 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RunVm, executionIndex: 0 >2014-08-11 18:59:48,522 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-49) Ending command successfully: org.ovirt.engine.core.bll.RunVmCommand >2014-08-11 18:59:48,550 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm completed, handling the result. >2014-08-11 18:59:48,551 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm succeeded, clearing tasks. >2014-08-11 18:59:48,563 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-49) SPMAsyncTask::ClearAsyncTask: Attempting to clear task f8414158-0ec0-4852-80cd-1f97eabe42eb >2014-08-11 18:59:48,564 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = f8414158-0ec0-4852-80cd-1f97eabe42eb), log id: 56dfc723 >2014-08-11 18:59:48,565 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=f8414158-0ec0-4852-80cd-1f97eabe42eb), log id: 7d8af763 >2014-08-11 18:59:48,659 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) FINISH, HSMClearTaskVDSCommand, log id: 7d8af763 >2014-08-11 18:59:48,660 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) FINISH, SPMClearTaskVDSCommand, log id: 56dfc723 >2014-08-11 18:59:48,687 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-49) BaseAsyncTask::removeTaskFromDB: Removed task f8414158-0ec0-4852-80cd-1f97eabe42eb from DataBase >2014-08-11 18:59:48,688 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-49) SPMAsyncTask::ClearAsyncTask: Attempting to clear task d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 >2014-08-11 18:59:48,689 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = d55803cf-0c5c-460d-a5cf-8c7705a3ebf0), log id: 246b330b >2014-08-11 18:59:48,691 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=d55803cf-0c5c-460d-a5cf-8c7705a3ebf0), log id: 79e429da >2014-08-11 18:59:48,743 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) FINISH, HSMClearTaskVDSCommand, log id: 79e429da >2014-08-11 18:59:48,744 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) FINISH, SPMClearTaskVDSCommand, log id: 246b330b >2014-08-11 18:59:48,754 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-49) BaseAsyncTask::removeTaskFromDB: Removed task d55803cf-0c5c-460d-a5cf-8c7705a3ebf0 from DataBase >2014-08-11 18:59:48,754 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity faae2f3e-ca48-49dd-9adc-aa5e63317ed4 >2014-08-11 19:01:17,470 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-74) Setting new tasks map. The map contains now 0 tasks >2014-08-11 19:01:17,472 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-74) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 19:04:15,625 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [2687b5ae] Lock Acquired to object EngineLock [exclusiveLocks= key: d4f5d124-ebf2-4c76-8630-1c56803547d7 value: VM >, sharedLocks= ] >2014-08-11 19:04:15,683 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [2687b5ae] START, IsVmDuringInitiatingVDSCommand( vmId = d4f5d124-ebf2-4c76-8630-1c56803547d7), log id: 780dac7a >2014-08-11 19:04:15,683 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [2687b5ae] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 780dac7a >2014-08-11 19:04:15,740 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-35) [2687b5ae] Running command: RunVmCommand internal: false. Entities affected : ID: d4f5d124-ebf2-4c76-8630-1c56803547d7 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 19:04:15,812 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-35) [2687b5ae] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@a467fbe0), log id: 4c49b5f0 >2014-08-11 19:04:15,823 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-35) [2687b5ae] FINISH, UpdateVmDynamicDataVDSCommand, log id: 4c49b5f0 >2014-08-11 19:04:15,826 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-35) [2687b5ae] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, vm=VM [vm5]), log id: 2820e9cb >2014-08-11 19:04:15,828 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-35) [2687b5ae] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, vm=VM [vm5]), log id: 5e1b82ee >2014-08-11 19:04:15,840 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-35) [2687b5ae] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7,acpiEnable=true,custom={device_af295ab2-0925-4ec0-b50f-6233f0f851c5device_5ee6b9fa-9423-4656-bcf6-ddc834730a60device_908f53d3-b5c3-4eb9-b3f4-b7801040e494device_791ba641-3591-4311-94f2-fe2a27049b86=VmDevice {vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, deviceId=791ba641-3591-4311-94f2-fe2a27049b86, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_af295ab2-0925-4ec0-b50f-6233f0f851c5device_5ee6b9fa-9423-4656-bcf6-ddc834730a60device_908f53d3-b5c3-4eb9-b3f4-b7801040e494=VmDevice {vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, deviceId=908f53d3-b5c3-4eb9-b3f4-b7801040e494, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_af295ab2-0925-4ec0-b50f-6233f0f851c5=VmDevice {vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, deviceId=af295ab2-0925-4ec0-b50f-6233f0f851c5, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_af295ab2-0925-4ec0-b50f-6233f0f851c5device_5ee6b9fa-9423-4656-bcf6-ddc834730a60=VmDevice {vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, deviceId=5ee6b9fa-9423-4656-bcf6-ddc834730a60, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={ram=65536, vram=32768, heads=1}, device=qxl, type=video, deviceId=38b3bf17-02de-4051-b54d-43bed8d06ad5}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={}, path=, device=cdrom, type=disk, readonly=true, deviceId=1b1ef765-07d1-4c87-a11f-64d95d7c6697}, {shared=false, index=0, volumeID=9d04820d-175e-45f3-9f24-11eb89a897cc, propagateErrors=off, format=raw, type=disk, iface=virtio, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageID=78dde0c0-a5d7-4e8c-bfe7-de91d182d512, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=78dde0c0-a5d7-4e8c-bfe7-de91d182d512}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:55, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=3fe2e63f-784f-41e9-a546-46fd19e5a56e}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=1cc7e9c9-a640-407d-88f4-27e00a73324d}, {address={bus=0x00, domain=0x0000, slot=0x05, type=pci, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=c31d509f-a3dc-405d-aa21-8c160e60cc86}],vmName=vm5,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 19:04:16,066 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-35) [2687b5ae] FINISH, CreateVDSCommand, log id: 5e1b82ee >2014-08-11 19:04:16,096 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-35) [2687b5ae] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 2820e9cb >2014-08-11 19:04:16,096 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-35) [2687b5ae] Lock freed to object EngineLock [exclusiveLocks= key: d4f5d124-ebf2-4c76-8630-1c56803547d7 value: VM >, sharedLocks= ] >2014-08-11 19:04:16,105 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-35) [2687b5ae] Correlation ID: 2687b5ae, Job ID: 5b0908d7-19e3-42cb-b72f-037ac3ded98d, Call Stack: null, Custom Event ID: -1, Message: VM vm5 was started by admin (Host: h1). >2014-08-11 19:04:17,510 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [340af3d6] Lock Acquired to object EngineLock [exclusiveLocks= key: 32f54e3c-df93-4be3-95c2-b843604932f9 value: VM >, sharedLocks= ] >2014-08-11 19:04:17,529 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [340af3d6] START, IsVmDuringInitiatingVDSCommand( vmId = 32f54e3c-df93-4be3-95c2-b843604932f9), log id: 2f059e51 >2014-08-11 19:04:17,529 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [340af3d6] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 2f059e51 >2014-08-11 19:04:17,612 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-46) [340af3d6] Running command: RunVmCommand internal: false. Entities affected : ID: 32f54e3c-df93-4be3-95c2-b843604932f9 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 19:04:17,651 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-46) [340af3d6] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@6aaa1143), log id: 6b9a28 >2014-08-11 19:04:17,662 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-46) [340af3d6] FINISH, UpdateVmDynamicDataVDSCommand, log id: 6b9a28 >2014-08-11 19:04:17,665 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-46) [340af3d6] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=32f54e3c-df93-4be3-95c2-b843604932f9, vm=VM [vm6]), log id: 2637584c >2014-08-11 19:04:17,667 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-46) [340af3d6] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=32f54e3c-df93-4be3-95c2-b843604932f9, vm=VM [vm6]), log id: 646f2b24 >2014-08-11 19:04:17,682 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-46) [340af3d6] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=32f54e3c-df93-4be3-95c2-b843604932f9,acpiEnable=true,custom={device_84c46df4-3569-42c7-a0b7-afabc1ce10a8device_117c8f36-5436-416e-a822-3e882d1572b8device_5b2690af-e7a4-4785-beca-92ec9ab1c3b2device_32cc4662-b24e-4cf2-b995-b3723c9f47aa=VmDevice {vmId=32f54e3c-df93-4be3-95c2-b843604932f9, deviceId=32cc4662-b24e-4cf2-b995-b3723c9f47aa, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_84c46df4-3569-42c7-a0b7-afabc1ce10a8=VmDevice {vmId=32f54e3c-df93-4be3-95c2-b843604932f9, deviceId=84c46df4-3569-42c7-a0b7-afabc1ce10a8, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_84c46df4-3569-42c7-a0b7-afabc1ce10a8device_117c8f36-5436-416e-a822-3e882d1572b8device_5b2690af-e7a4-4785-beca-92ec9ab1c3b2=VmDevice {vmId=32f54e3c-df93-4be3-95c2-b843604932f9, deviceId=5b2690af-e7a4-4785-beca-92ec9ab1c3b2, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_84c46df4-3569-42c7-a0b7-afabc1ce10a8device_117c8f36-5436-416e-a822-3e882d1572b8=VmDevice {vmId=32f54e3c-df93-4be3-95c2-b843604932f9, deviceId=117c8f36-5436-416e-a822-3e882d1572b8, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x06, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_84c46df4-3569-42c7-a0b7-afabc1ce10a8device_117c8f36-5436-416e-a822-3e882d1572b8device_5b2690af-e7a4-4785-beca-92ec9ab1c3b2device_32cc4662-b24e-4cf2-b995-b3723c9f47aadevice_f9b16f2c-1d10-4c04-8051-e0021a8157e2=VmDevice {vmId=32f54e3c-df93-4be3-95c2-b843604932f9, deviceId=f9b16f2c-1d10-4c04-8051-e0021a8157e2, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={ram=65536, vram=32768, heads=1}, device=qxl, type=video, deviceId=625180e7-3424-4834-a1fe-854a58e764b7}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=a3fbd6a6-6bad-4e03-8249-a7e2b756bc3b}, {shared=false, index=0, volumeID=9b979186-4a5f-4fe0-8233-b8c4778f76cb, propagateErrors=off, format=raw, type=disk, iface=virtio, bootOrder=1, address={bus=0x00, domain=0x0000, slot=0x05, type=pci, function=0x0}, domainID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageID=13d602b9-2253-4fd0-9409-813c8007cf8f, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=13d602b9-2253-4fd0-9409-813c8007cf8f}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:56, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=88428def-d086-486e-b6c8-3e841e354652}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=e155bde5-a92b-451f-b665-34c3da57a619}, {address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=117c8f36-5436-416e-a822-3e882d1572b8}],vmName=vm6,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 19:04:17,711 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-46) [340af3d6] FINISH, CreateVDSCommand, log id: 646f2b24 >2014-08-11 19:04:17,728 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-46) [340af3d6] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 2637584c >2014-08-11 19:04:17,729 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-46) [340af3d6] Lock freed to object EngineLock [exclusiveLocks= key: 32f54e3c-df93-4be3-95c2-b843604932f9 value: VM >, sharedLocks= ] >2014-08-11 19:04:17,737 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) [340af3d6] Correlation ID: 340af3d6, Job ID: f776f3c0-8ecd-46b5-8450-4bb429c52307, Call Stack: null, Custom Event ID: -1, Message: VM vm6 was started by admin (Host: h1). >2014-08-11 19:04:19,286 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) [1aa273ac] VM vm6 32f54e3c-df93-4be3-95c2-b843604932f9 moved from WaitForLaunch --> PoweringUp >2014-08-11 19:04:19,287 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) [1aa273ac] VM vm5 d4f5d124-ebf2-4c76-8630-1c56803547d7 moved from WaitForLaunch --> PoweringUp >2014-08-11 19:04:19,287 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-23) [1aa273ac] START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[32f54e3c-df93-4be3-95c2-b843604932f9, d4f5d124-ebf2-4c76-8630-1c56803547d7]), log id: 604b14f8 >2014-08-11 19:04:19,303 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-23) [1aa273ac] FINISH, FullListVdsCommand, return: [{displaySecurePort=5905, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.35.0.105, pauseCode=NOERR, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, acpiEnable=true, custom={device_af295ab2-0925-4ec0-b50f-6233f0f851c5device_5ee6b9fa-9423-4656-bcf6-ddc834730a60device_908f53d3-b5c3-4eb9-b3f4-b7801040e494=VmDevice {vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, deviceId=908f53d3-b5c3-4eb9-b3f4-b7801040e494, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_af295ab2-0925-4ec0-b50f-6233f0f851c5device_5ee6b9fa-9423-4656-bcf6-ddc834730a60device_908f53d3-b5c3-4eb9-b3f4-b7801040e494device_791ba641-3591-4311-94f2-fe2a27049b86=VmDevice {vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, deviceId=791ba641-3591-4311-94f2-fe2a27049b86, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_af295ab2-0925-4ec0-b50f-6233f0f851c5=VmDevice {vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, deviceId=af295ab2-0925-4ec0-b50f-6233f0f851c5, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_af295ab2-0925-4ec0-b50f-6233f0f851c5device_5ee6b9fa-9423-4656-bcf6-ddc834730a60=VmDevice {vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, deviceId=5ee6b9fa-9423-4656-bcf6-ddc834730a60, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=29025, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@4a29bd27, vmName=vm5, fileTransferEnable=true, cpuType=SandyBridge}, {displaySecurePort=5906, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.35.0.105, pauseCode=NOERR, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=32f54e3c-df93-4be3-95c2-b843604932f9, acpiEnable=true, custom={device_84c46df4-3569-42c7-a0b7-afabc1ce10a8device_117c8f36-5436-416e-a822-3e882d1572b8device_5b2690af-e7a4-4785-beca-92ec9ab1c3b2device_32cc4662-b24e-4cf2-b995-b3723c9f47aa=VmDevice {vmId=32f54e3c-df93-4be3-95c2-b843604932f9, deviceId=32cc4662-b24e-4cf2-b995-b3723c9f47aa, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_84c46df4-3569-42c7-a0b7-afabc1ce10a8=VmDevice {vmId=32f54e3c-df93-4be3-95c2-b843604932f9, deviceId=84c46df4-3569-42c7-a0b7-afabc1ce10a8, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_84c46df4-3569-42c7-a0b7-afabc1ce10a8device_117c8f36-5436-416e-a822-3e882d1572b8=VmDevice {vmId=32f54e3c-df93-4be3-95c2-b843604932f9, deviceId=117c8f36-5436-416e-a822-3e882d1572b8, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x06, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_84c46df4-3569-42c7-a0b7-afabc1ce10a8device_117c8f36-5436-416e-a822-3e882d1572b8device_5b2690af-e7a4-4785-beca-92ec9ab1c3b2=VmDevice {vmId=32f54e3c-df93-4be3-95c2-b843604932f9, deviceId=5b2690af-e7a4-4785-beca-92ec9ab1c3b2, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_84c46df4-3569-42c7-a0b7-afabc1ce10a8device_117c8f36-5436-416e-a822-3e882d1572b8device_5b2690af-e7a4-4785-beca-92ec9ab1c3b2device_32cc4662-b24e-4cf2-b995-b3723c9f47aadevice_f9b16f2c-1d10-4c04-8051-e0021a8157e2=VmDevice {vmId=32f54e3c-df93-4be3-95c2-b843604932f9, deviceId=f9b16f2c-1d10-4c04-8051-e0021a8157e2, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=29150, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@5e813f4a, vmName=vm6, fileTransferEnable=true, cpuType=SandyBridge}], log id: 604b14f8 >2014-08-11 19:04:19,312 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) [1aa273ac] Received a memballoon Device without an address when processing VM d4f5d124-ebf2-4c76-8630-1c56803547d7 devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-08-11 19:04:19,313 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) [1aa273ac] Received a spice Device without an address when processing VM d4f5d124-ebf2-4c76-8630-1c56803547d7 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5905, type=graphics} >2014-08-11 19:04:19,315 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) [1aa273ac] Received a memballoon Device without an address when processing VM 32f54e3c-df93-4be3-95c2-b843604932f9 devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-08-11 19:04:19,316 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) [1aa273ac] Received a spice Device without an address when processing VM 32f54e3c-df93-4be3-95c2-b843604932f9 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5906, type=graphics} >2014-08-11 19:04:29,257 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [3065b0a6] Lock Acquired to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-11 19:04:29,300 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [3065b0a6] START, IsVmDuringInitiatingVDSCommand( vmId = 337241af-a081-4971-a3d2-4c7a790e030d), log id: b20aed4 >2014-08-11 19:04:29,300 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [3065b0a6] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: b20aed4 >2014-08-11 19:04:29,382 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-6) [3065b0a6] Running command: RunVmCommand internal: false. Entities affected : ID: 337241af-a081-4971-a3d2-4c7a790e030d Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 19:04:29,422 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-6) [3065b0a6] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@b6d25946), log id: 74c563b >2014-08-11 19:04:29,431 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-6) [3065b0a6] FINISH, UpdateVmDynamicDataVDSCommand, log id: 74c563b >2014-08-11 19:04:29,438 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-6) [3065b0a6] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=337241af-a081-4971-a3d2-4c7a790e030d, vm=VM [vm2]), log id: 6ecf3fe8 >2014-08-11 19:04:29,441 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-6) [3065b0a6] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=337241af-a081-4971-a3d2-4c7a790e030d, vm=VM [vm2]), log id: 41888375 >2014-08-11 19:04:29,452 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-6) [3065b0a6] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=337241af-a081-4971-a3d2-4c7a790e030d,acpiEnable=true,custom={device_8d638782-8733-47c4-9741-5a1e92d08372device_888154a0-4405-4313-8180-8d6be0599b3ddevice_44a1b1c2-fa39-474f-90f6-8f1b60b3fc93device_9fdf2a63-39b7-4b67-82b6-480dbda751a1=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=9fdf2a63-39b7-4b67-82b6-480dbda751a1, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_8d638782-8733-47c4-9741-5a1e92d08372device_888154a0-4405-4313-8180-8d6be0599b3d=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=888154a0-4405-4313-8180-8d6be0599b3d, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_8d638782-8733-47c4-9741-5a1e92d08372device_888154a0-4405-4313-8180-8d6be0599b3ddevice_44a1b1c2-fa39-474f-90f6-8f1b60b3fc93=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=44a1b1c2-fa39-474f-90f6-8f1b60b3fc93, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_8d638782-8733-47c4-9741-5a1e92d08372=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=8d638782-8733-47c4-9741-5a1e92d08372, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=2d206443-6c86-4c8a-9e3b-7d8ada7cdcee}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={}, path=, device=cdrom, type=disk, readonly=true, deviceId=c3bd31fd-43e6-4512-8cae-cd0e9f7b0894}, {shared=false, index=0, volumeID=4024f2e3-f164-402a-b49d-9305398a0aeb, propagateErrors=off, format=raw, type=disk, iface=virtio, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageID=2b033ab4-8cd6-4214-baf6-1252264f371e, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=2b033ab4-8cd6-4214-baf6-1252264f371e}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:52, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=b4520306-938e-4534-a3a2-04d9242b8747}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=b5d31f8f-6bd4-4cea-9015-4b05581b9e67}, {address={bus=0x00, domain=0x0000, slot=0x05, type=pci, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=536eef98-f402-4af6-bc12-8fe6eb620c37}],vmName=vm2,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 19:04:29,480 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-6) [3065b0a6] FINISH, CreateVDSCommand, log id: 41888375 >2014-08-11 19:04:29,498 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-6) [3065b0a6] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 6ecf3fe8 >2014-08-11 19:04:29,498 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-6) [3065b0a6] Lock freed to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-11 19:04:29,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-6) [3065b0a6] Correlation ID: 3065b0a6, Job ID: 89e6dea5-4fba-4d0a-8aa5-914aed9c2a75, Call Stack: null, Custom Event ID: -1, Message: VM vm2 was started by admin (Host: h1). >2014-08-11 19:04:31,820 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) VM vm2 337241af-a081-4971-a3d2-4c7a790e030d moved from WaitForLaunch --> PoweringUp >2014-08-11 19:04:31,820 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-38) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[337241af-a081-4971-a3d2-4c7a790e030d]), log id: 68a18570 >2014-08-11 19:04:31,867 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-38) FINISH, FullListVdsCommand, return: [{displaySecurePort=5907, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.35.0.105, pauseCode=NOERR, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=337241af-a081-4971-a3d2-4c7a790e030d, acpiEnable=true, custom={device_8d638782-8733-47c4-9741-5a1e92d08372device_888154a0-4405-4313-8180-8d6be0599b3d=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=888154a0-4405-4313-8180-8d6be0599b3d, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_8d638782-8733-47c4-9741-5a1e92d08372device_888154a0-4405-4313-8180-8d6be0599b3ddevice_44a1b1c2-fa39-474f-90f6-8f1b60b3fc93device_9fdf2a63-39b7-4b67-82b6-480dbda751a1=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=9fdf2a63-39b7-4b67-82b6-480dbda751a1, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_8d638782-8733-47c4-9741-5a1e92d08372device_888154a0-4405-4313-8180-8d6be0599b3ddevice_44a1b1c2-fa39-474f-90f6-8f1b60b3fc93=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=44a1b1c2-fa39-474f-90f6-8f1b60b3fc93, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_8d638782-8733-47c4-9741-5a1e92d08372=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=8d638782-8733-47c4-9741-5a1e92d08372, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=29292, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@33b53c4e, vmName=vm2, fileTransferEnable=true, cpuType=SandyBridge}], log id: 68a18570 >2014-08-11 19:04:31,873 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Received a memballoon Device without an address when processing VM 337241af-a081-4971-a3d2-4c7a790e030d devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-08-11 19:04:31,874 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Received a spice Device without an address when processing VM 337241af-a081-4971-a3d2-4c7a790e030d devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5907, type=graphics} >2014-08-11 19:05:18,499 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) VM vm6 32f54e3c-df93-4be3-95c2-b843604932f9 moved from PoweringUp --> Up >2014-08-11 19:05:18,500 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) VM vm5 d4f5d124-ebf2-4c76-8630-1c56803547d7 moved from PoweringUp --> Up >2014-08-11 19:05:18,557 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) Correlation ID: 340af3d6, Job ID: f776f3c0-8ecd-46b5-8450-4bb429c52307, Call Stack: null, Custom Event ID: -1, Message: VM vm6 started on Host h1 >2014-08-11 19:05:18,582 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) Correlation ID: 2687b5ae, Job ID: 5b0908d7-19e3-42cb-b72f-037ac3ded98d, Call Stack: null, Custom Event ID: -1, Message: VM vm5 started on Host h1 >2014-08-11 19:05:30,841 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) VM vm2 337241af-a081-4971-a3d2-4c7a790e030d moved from PoweringUp --> Up >2014-08-11 19:05:30,964 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-56) Correlation ID: 3065b0a6, Job ID: 89e6dea5-4fba-4d0a-8aa5-914aed9c2a75, Call Stack: null, Custom Event ID: -1, Message: VM vm2 started on Host h1 >2014-08-11 19:19:02,146 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [3d693f6] Lock Acquired to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-11 19:19:02,253 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-36) [3d693f6] Running command: HibernateVmCommand internal: false. Entities affected : ID: 337241af-a081-4971-a3d2-4c7a790e030d Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 19:19:02,259 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-36) [3d693f6] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@962017b4), log id: 3e38897e >2014-08-11 19:19:02,261 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-36) [3d693f6] FINISH, UpdateVmDynamicDataVDSCommand, log id: 3e38897e >2014-08-11 19:19:02,272 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-36) [3d693f6] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 0b184d81-6c16-457b-92ab-397339be66b5, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = e4388048-bcc4-4469-82d7-2ce1e7d58623, newImageDescription = ), log id: 1c138d55 >2014-08-11 19:19:02,274 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-36) [3d693f6] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 19:19:02,640 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-36) [3d693f6] FINISH, CreateImageVDSCommand, return: e4388048-bcc4-4469-82d7-2ce1e7d58623, log id: 1c138d55 >2014-08-11 19:19:02,652 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-36) [3d693f6] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 31c87499-2fce-4370-a0fd-ea0ddeb8b5d6 >2014-08-11 19:19:02,653 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-36) [3d693f6] CommandMultiAsyncTasks::AttachTask: Attaching task 46efd818-7467-47d1-85cf-8ae0a96e6f06 to command 31c87499-2fce-4370-a0fd-ea0ddeb8b5d6. >2014-08-11 19:19:02,671 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-36) [3d693f6] Adding task 46efd818-7467-47d1-85cf-8ae0a96e6f06 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 19:19:02,701 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-36) [3d693f6] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 6f1a3aee-11ac-468c-bbd0-8e10014896c7, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = d0d54fd8-2374-4084-94dc-04552044bbed, newImageDescription = ), log id: 343c72cd >2014-08-11 19:19:02,707 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-36) [3d693f6] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-11 19:19:03,220 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-36) [3d693f6] FINISH, CreateImageVDSCommand, return: d0d54fd8-2374-4084-94dc-04552044bbed, log id: 343c72cd >2014-08-11 19:19:03,232 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-36) [3d693f6] CommandMultiAsyncTasks::AttachTask: Attaching task fd82f057-b995-4b15-8de0-6a87a17ecccd to command 31c87499-2fce-4370-a0fd-ea0ddeb8b5d6. >2014-08-11 19:19:03,247 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-36) [3d693f6] Adding task fd82f057-b995-4b15-8de0-6a87a17ecccd (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 19:19:03,256 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-36) [3d693f6] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@be6b1070), log id: 7bf46009 >2014-08-11 19:19:03,263 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-36) [3d693f6] FINISH, UpdateVmDynamicDataVDSCommand, log id: 7bf46009 >2014-08-11 19:19:03,289 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-36) [3d693f6] Correlation ID: 3d693f6, Job ID: d45b159e-32a8-48df-8e16-0f027e8e11b9, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm2 was initiated by User admin (Host: h1). >2014-08-11 19:19:03,289 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-36) [3d693f6] BaseAsyncTask::startPollingTask: Starting to poll task 46efd818-7467-47d1-85cf-8ae0a96e6f06. >2014-08-11 19:19:03,290 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-36) [3d693f6] BaseAsyncTask::startPollingTask: Starting to poll task fd82f057-b995-4b15-8de0-6a87a17ecccd. >2014-08-11 19:19:03,290 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-36) [3d693f6] Lock freed to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-11 19:19:08,527 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-25) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 19:19:08,535 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-25) SPMAsyncTask::PollTask: Polling task 46efd818-7467-47d1-85cf-8ae0a96e6f06 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 19:19:08,599 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-25) BaseAsyncTask::onTaskEndSuccess: Task 46efd818-7467-47d1-85cf-8ae0a96e6f06 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 19:19:08,600 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-25) Task with DB Task ID e0c7e53b-1ee8-45ce-af71-c89544d8234b and VDSM Task ID fd82f057-b995-4b15-8de0-6a87a17ecccd is in state Polling. End action for command 31c87499-2fce-4370-a0fd-ea0ddeb8b5d6 will proceed when all the entitys tasks are completed. >2014-08-11 19:19:08,601 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-25) SPMAsyncTask::PollTask: Polling task fd82f057-b995-4b15-8de0-6a87a17ecccd (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 19:19:08,628 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-25) BaseAsyncTask::onTaskEndSuccess: Task fd82f057-b995-4b15-8de0-6a87a17ecccd (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 19:19:08,629 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-25) CommandAsyncTask::endActionIfNecessary: All tasks of command 31c87499-2fce-4370-a0fd-ea0ddeb8b5d6 has ended -> executing endAction >2014-08-11 19:19:08,630 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-25) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 31c87499-2fce-4370-a0fd-ea0ddeb8b5d6): calling endAction . >2014-08-11 19:19:08,631 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-11 19:19:08,665 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-14) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-11 19:19:08,666 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=337241af-a081-4971-a3d2-4c7a790e030d, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,0b184d81-6c16-457b-92ab-397339be66b5,e4388048-bcc4-4469-82d7-2ce1e7d58623,6f1a3aee-11ac-468c-bbd0-8e10014896c7,d0d54fd8-2374-4084-94dc-04552044bbed), log id: eaad304 >2014-08-11 19:19:08,668 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=337241af-a081-4971-a3d2-4c7a790e030d, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,0b184d81-6c16-457b-92ab-397339be66b5,e4388048-bcc4-4469-82d7-2ce1e7d58623,6f1a3aee-11ac-468c-bbd0-8e10014896c7,d0d54fd8-2374-4084-94dc-04552044bbed), log id: a78f539 >2014-08-11 19:19:08,675 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, HibernateBrokerVDSCommand, log id: a78f539 >2014-08-11 19:19:08,681 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, HibernateVDSCommand, log id: eaad304 >2014-08-11 19:19:08,683 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-11 19:19:08,683 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-11 19:19:08,689 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) SPMAsyncTask::ClearAsyncTask: Attempting to clear task fd82f057-b995-4b15-8de0-6a87a17ecccd >2014-08-11 19:19:08,690 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = fd82f057-b995-4b15-8de0-6a87a17ecccd), log id: 3beeeb37 >2014-08-11 19:19:08,715 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=fd82f057-b995-4b15-8de0-6a87a17ecccd), log id: 5336852b >2014-08-11 19:19:08,804 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, HSMClearTaskVDSCommand, log id: 5336852b >2014-08-11 19:19:08,804 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, SPMClearTaskVDSCommand, log id: 3beeeb37 >2014-08-11 19:19:08,813 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) BaseAsyncTask::removeTaskFromDB: Removed task fd82f057-b995-4b15-8de0-6a87a17ecccd from DataBase >2014-08-11 19:19:08,814 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 46efd818-7467-47d1-85cf-8ae0a96e6f06 >2014-08-11 19:19:08,814 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 46efd818-7467-47d1-85cf-8ae0a96e6f06), log id: 20b52a0c >2014-08-11 19:19:08,816 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=46efd818-7467-47d1-85cf-8ae0a96e6f06), log id: 55db8a9a >2014-08-11 19:19:08,886 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, HSMClearTaskVDSCommand, log id: 55db8a9a >2014-08-11 19:19:08,886 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) FINISH, SPMClearTaskVDSCommand, log id: 20b52a0c >2014-08-11 19:19:08,896 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) BaseAsyncTask::removeTaskFromDB: Removed task 46efd818-7467-47d1-85cf-8ae0a96e6f06 from DataBase >2014-08-11 19:19:08,897 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 31c87499-2fce-4370-a0fd-ea0ddeb8b5d6 >2014-08-11 19:19:12,147 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-19) START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=337241af-a081-4971-a3d2-4c7a790e030d, force=false, secondsToWait=0, gracefully=false, reason=), log id: 1f4eb48c >2014-08-11 19:19:12,158 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, DestroyVDSCommand, log id: 1f4eb48c >2014-08-11 19:19:12,234 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm2 on Host h1 is suspended. >2014-08-11 19:19:12,236 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) VM vm2 (337241af-a081-4971-a3d2-4c7a790e030d) is running in db and not running in VDS h1 >2014-08-11 19:19:12,236 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-19) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[337241af-a081-4971-a3d2-4c7a790e030d]), log id: 76d2b758 >2014-08-11 19:19:12,240 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-19) FINISH, FullListVdsCommand, return: [], log id: 76d2b758 >2014-08-11 19:19:12,272 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-10) [42790bbc] Running command: ProcessDownVmCommand internal: true. >2014-08-11 19:19:26,454 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-2) [3fdff197] Lock Acquired to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-11 19:19:26,506 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [3fdff197] START, IsVmDuringInitiatingVDSCommand( vmId = 337241af-a081-4971-a3d2-4c7a790e030d), log id: 177aaecc >2014-08-11 19:19:26,507 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [3fdff197] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 177aaecc >2014-08-11 19:19:26,556 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-16) [3fdff197] Running command: RunVmCommand internal: false. Entities affected : ID: 337241af-a081-4971-a3d2-4c7a790e030d Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 19:19:26,604 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-16) [3fdff197] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@d37b9b35), log id: 2a93318e >2014-08-11 19:19:26,616 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-16) [3fdff197] FINISH, UpdateVmDynamicDataVDSCommand, log id: 2a93318e >2014-08-11 19:19:26,618 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-16) [3fdff197] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=337241af-a081-4971-a3d2-4c7a790e030d, vm=VM [vm2]), log id: 5dbb8e72 >2014-08-11 19:19:26,620 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-16) [3fdff197] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=337241af-a081-4971-a3d2-4c7a790e030d, vm=VM [vm2]), log id: 36481a2d >2014-08-11 19:19:26,633 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-16) [3fdff197] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,hiberVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,0b184d81-6c16-457b-92ab-397339be66b5,e4388048-bcc4-4469-82d7-2ce1e7d58623,6f1a3aee-11ac-468c-bbd0-8e10014896c7,d0d54fd8-2374-4084-94dc-04552044bbed,transparentHugePages=true,vmId=337241af-a081-4971-a3d2-4c7a790e030d,acpiEnable=true,custom={device_222af50b-7995-41a4-a438-2a348a72123ddevice_217d9e53-8e78-48c5-af28-bf4ef5cf02a8device_e995eb39-4cb6-4c40-9be5-716988c153fb=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=e995eb39-4cb6-4c40-9be5-716988c153fb, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_222af50b-7995-41a4-a438-2a348a72123d=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=222af50b-7995-41a4-a438-2a348a72123d, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_222af50b-7995-41a4-a438-2a348a72123ddevice_217d9e53-8e78-48c5-af28-bf4ef5cf02a8device_e995eb39-4cb6-4c40-9be5-716988c153fbdevice_e7b9fbdb-52a2-4a6d-bc92-a3947ef2e8f1=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=e7b9fbdb-52a2-4a6d-bc92-a3947ef2e8f1, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_222af50b-7995-41a4-a438-2a348a72123ddevice_217d9e53-8e78-48c5-af28-bf4ef5cf02a8=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=217d9e53-8e78-48c5-af28-bf4ef5cf02a8, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=2d206443-6c86-4c8a-9e3b-7d8ada7cdcee}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={}, path=, device=cdrom, type=disk, readonly=true, deviceId=c3bd31fd-43e6-4512-8cae-cd0e9f7b0894}, {shared=false, index=0, volumeID=4024f2e3-f164-402a-b49d-9305398a0aeb, propagateErrors=off, format=raw, type=disk, iface=virtio, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageID=2b033ab4-8cd6-4214-baf6-1252264f371e, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=2b033ab4-8cd6-4214-baf6-1252264f371e}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:52, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=b4520306-938e-4534-a3a2-04d9242b8747}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=b5d31f8f-6bd4-4cea-9015-4b05581b9e67}, {address={bus=0x00, domain=0x0000, slot=0x05, type=pci, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=536eef98-f402-4af6-bc12-8fe6eb620c37}],vmName=vm2,cpuType=SandyBridge,fileTransferEnable=true >2014-08-11 19:19:26,711 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-16) [3fdff197] FINISH, CreateVDSCommand, log id: 36481a2d >2014-08-11 19:19:26,732 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-16) [3fdff197] FINISH, CreateVmVDSCommand, return: RestoringState, log id: 5dbb8e72 >2014-08-11 19:19:26,732 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-16) [3fdff197] Lock freed to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-11 19:19:26,740 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-16) [3fdff197] Correlation ID: 3fdff197, Job ID: 03b187d1-2914-4f45-a617-d7192cf05053, Call Stack: null, Custom Event ID: -1, Message: VM vm2 was started by admin (Host: h1). >2014-08-11 19:19:27,497 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) VM vm2 337241af-a081-4971-a3d2-4c7a790e030d moved from RestoringState --> Up >2014-08-11 19:19:27,498 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-44) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[337241af-a081-4971-a3d2-4c7a790e030d]), log id: 664a33ff >2014-08-11 19:19:27,508 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-44) FINISH, FullListVdsCommand, return: [{displaySecurePort=5907, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=337241af-a081-4971-a3d2-4c7a790e030d, acpiEnable=true, custom={device_8d638782-8733-47c4-9741-5a1e92d08372device_888154a0-4405-4313-8180-8d6be0599b3d=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=888154a0-4405-4313-8180-8d6be0599b3d, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_8d638782-8733-47c4-9741-5a1e92d08372device_888154a0-4405-4313-8180-8d6be0599b3ddevice_44a1b1c2-fa39-474f-90f6-8f1b60b3fc93device_9fdf2a63-39b7-4b67-82b6-480dbda751a1=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=9fdf2a63-39b7-4b67-82b6-480dbda751a1, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_8d638782-8733-47c4-9741-5a1e92d08372device_888154a0-4405-4313-8180-8d6be0599b3ddevice_44a1b1c2-fa39-474f-90f6-8f1b60b3fc93=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=44a1b1c2-fa39-474f-90f6-8f1b60b3fc93, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_8d638782-8733-47c4-9741-5a1e92d08372=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=8d638782-8733-47c4-9741-5a1e92d08372, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=30648, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@119bded0, vmName=vm2, fileTransferEnable=true, cpuType=SandyBridge}], log id: 664a33ff >2014-08-11 19:19:27,515 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Received a memballoon Device without an address when processing VM 337241af-a081-4971-a3d2-4c7a790e030d devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 19:19:27,516 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Received a spice Device without an address when processing VM 337241af-a081-4971-a3d2-4c7a790e030d devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5907, type=graphics} >2014-08-11 19:19:27,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-44) Correlation ID: 3fdff197, Job ID: 03b187d1-2914-4f45-a617-d7192cf05053, Call Stack: null, Custom Event ID: -1, Message: VM vm2 started on Host h1 >2014-08-11 19:19:27,660 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (DefaultQuartzScheduler_Worker-44) [1db3e926] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-11 19:19:27,717 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-44) [1db3e926] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 0b184d81-6c16-457b-92ab-397339be66b5, postZeros = false, forceDelete = false), log id: 2e4af984 >2014-08-11 19:19:28,186 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-44) [1db3e926] FINISH, DeleteImageGroupVDSCommand, log id: 2e4af984 >2014-08-11 19:19:28,198 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-44) [1db3e926] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command a244c35d-0f2c-4d89-8296-7b7197813a17 >2014-08-11 19:19:28,199 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-44) [1db3e926] CommandMultiAsyncTasks::AttachTask: Attaching task ad97eeef-e100-4acb-9b01-bf8290fdcaca to command a244c35d-0f2c-4d89-8296-7b7197813a17. >2014-08-11 19:19:28,214 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-44) [1db3e926] Adding task ad97eeef-e100-4acb-9b01-bf8290fdcaca (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 19:19:28,239 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-44) [1db3e926] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 6f1a3aee-11ac-468c-bbd0-8e10014896c7, postZeros = false, forceDelete = false), log id: 1195f248 >2014-08-11 19:19:28,774 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-44) [1db3e926] FINISH, DeleteImageGroupVDSCommand, log id: 1195f248 >2014-08-11 19:19:28,811 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-44) [1db3e926] CommandMultiAsyncTasks::AttachTask: Attaching task 580d0300-9d90-4abc-9558-565d9009b1d0 to command a244c35d-0f2c-4d89-8296-7b7197813a17. >2014-08-11 19:19:28,835 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-44) [1db3e926] Adding task 580d0300-9d90-4abc-9558-565d9009b1d0 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 19:19:28,861 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-44) [1db3e926] BaseAsyncTask::startPollingTask: Starting to poll task ad97eeef-e100-4acb-9b01-bf8290fdcaca. >2014-08-11 19:19:28,862 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-44) [1db3e926] BaseAsyncTask::startPollingTask: Starting to poll task 580d0300-9d90-4abc-9558-565d9009b1d0. >2014-08-11 19:19:28,863 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-44) [1db3e926] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@8404dd2f), log id: 2b2642a1 >2014-08-11 19:19:28,868 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-44) [1db3e926] FINISH, UpdateVmDynamicDataVDSCommand, log id: 2b2642a1 >2014-08-11 19:19:31,555 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [4692193c] Lock Acquired to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-11 19:19:31,555 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [4692193c] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:255) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor394.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 19:19:31,634 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [4692193c] Lock freed to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-11 19:19:31,906 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-57) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[337241af-a081-4971-a3d2-4c7a790e030d]), log id: 7698fd41 >2014-08-11 19:19:31,916 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-57) FINISH, FullListVdsCommand, return: [{displaySecurePort=5907, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=337241af-a081-4971-a3d2-4c7a790e030d, acpiEnable=true, custom={device_8d638782-8733-47c4-9741-5a1e92d08372device_888154a0-4405-4313-8180-8d6be0599b3d=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=888154a0-4405-4313-8180-8d6be0599b3d, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_8d638782-8733-47c4-9741-5a1e92d08372device_888154a0-4405-4313-8180-8d6be0599b3ddevice_44a1b1c2-fa39-474f-90f6-8f1b60b3fc93device_9fdf2a63-39b7-4b67-82b6-480dbda751a1=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=9fdf2a63-39b7-4b67-82b6-480dbda751a1, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_8d638782-8733-47c4-9741-5a1e92d08372device_888154a0-4405-4313-8180-8d6be0599b3ddevice_44a1b1c2-fa39-474f-90f6-8f1b60b3fc93=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=44a1b1c2-fa39-474f-90f6-8f1b60b3fc93, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_8d638782-8733-47c4-9741-5a1e92d08372=VmDevice {vmId=337241af-a081-4971-a3d2-4c7a790e030d, deviceId=8d638782-8733-47c4-9741-5a1e92d08372, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=30648, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@5ca10e5e, vmName=vm2, fileTransferEnable=true, cpuType=SandyBridge}], log id: 7698fd41 >2014-08-11 19:19:31,923 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Received a memballoon Device without an address when processing VM 337241af-a081-4971-a3d2-4c7a790e030d devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-11 19:19:31,925 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Received a spice Device without an address when processing VM 337241af-a081-4971-a3d2-4c7a790e030d devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5907, type=graphics} >2014-08-11 19:19:38,631 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-68) task id 580d0300-9d90-4abc-9558-565d9009b1d0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:19:38,633 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-68) task id ad97eeef-e100-4acb-9b01-bf8290fdcaca is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:19:39,422 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [7d810455] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-11 19:19:39,423 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [7d810455] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:255) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor394.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 19:19:39,447 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [7d810455] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-11 19:19:46,721 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [3d3e1c6] Lock Acquired to object EngineLock [exclusiveLocks= key: de580b19-e975-426f-80de-6c7daf1f24ab value: VM >, sharedLocks= ] >2014-08-11 19:19:46,721 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [3d3e1c6] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:255) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor394.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 19:19:46,747 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [3d3e1c6] Lock freed to object EngineLock [exclusiveLocks= key: de580b19-e975-426f-80de-6c7daf1f24ab value: VM >, sharedLocks= ] >2014-08-11 19:19:48,634 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) task id 580d0300-9d90-4abc-9558-565d9009b1d0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:19:48,635 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) task id ad97eeef-e100-4acb-9b01-bf8290fdcaca is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:19:51,958 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [304854ee] Lock Acquired to object EngineLock [exclusiveLocks= key: d4f5d124-ebf2-4c76-8630-1c56803547d7 value: VM >, sharedLocks= ] >2014-08-11 19:19:51,959 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [304854ee] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:255) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor394.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 19:19:51,985 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [304854ee] Lock freed to object EngineLock [exclusiveLocks= key: d4f5d124-ebf2-4c76-8630-1c56803547d7 value: VM >, sharedLocks= ] >2014-08-11 19:19:57,311 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [55d944a5] Lock Acquired to object EngineLock [exclusiveLocks= key: 32f54e3c-df93-4be3-95c2-b843604932f9 value: VM >, sharedLocks= ] >2014-08-11 19:19:57,312 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [55d944a5] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.TaskManagerUtil.entityHasTasks(TaskManagerUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:255) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor394.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-11 19:19:57,338 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-2) [55d944a5] Lock freed to object EngineLock [exclusiveLocks= key: 32f54e3c-df93-4be3-95c2-b843604932f9 value: VM >, sharedLocks= ] >2014-08-11 19:19:58,635 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-97) task id 580d0300-9d90-4abc-9558-565d9009b1d0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:19:58,635 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-97) task id ad97eeef-e100-4acb-9b01-bf8290fdcaca is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:08,636 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-19) task id 580d0300-9d90-4abc-9558-565d9009b1d0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:08,637 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-19) task id ad97eeef-e100-4acb-9b01-bf8290fdcaca is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:09,919 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-11) [4d1a11c8] Running command: StopVmCommand internal: false. Entities affected : ID: 337241af-a081-4971-a3d2-4c7a790e030d Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 19:20:09,923 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-11) [4d1a11c8] START, DestroyVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=337241af-a081-4971-a3d2-4c7a790e030d, force=false, secondsToWait=0, gracefully=false, reason=), log id: 34d645ec >2014-08-11 19:20:09,927 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-11) [4d1a11c8] START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=337241af-a081-4971-a3d2-4c7a790e030d, force=false, secondsToWait=0, gracefully=false, reason=), log id: 40bcb24e >2014-08-11 19:20:10,210 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-11) [4d1a11c8] FINISH, DestroyVDSCommand, log id: 40bcb24e >2014-08-11 19:20:10,225 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-11) [4d1a11c8] FINISH, DestroyVmVDSCommand, return: Down, log id: 34d645ec >2014-08-11 19:20:10,228 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-29) [132befb5] Running command: ProcessDownVmCommand internal: true. >2014-08-11 19:20:10,242 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-11) [4d1a11c8] Correlation ID: 4d1a11c8, Job ID: 33e89429-e64c-45b7-97f8-9fe7edf38754, Call Stack: null, Custom Event ID: -1, Message: VM vm2 powered off by admin (Host: h1) (Reason: Not Specified). >2014-08-11 19:20:17,075 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-25) [7ae86507] Running command: StopVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 19:20:17,081 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-25) [7ae86507] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@65fa0423), log id: 77f32f65 >2014-08-11 19:20:17,083 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-25) [7ae86507] FINISH, UpdateVmDynamicDataVDSCommand, log id: 77f32f65 >2014-08-11 19:20:17,095 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-11 19:20:17,125 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 4c2b3c72-a901-435d-9a9f-8628b2263c7e, postZeros = false, forceDelete = false), log id: 37482eaa >2014-08-11 19:20:17,492 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-39) Setting new tasks map. The map contains now 2 tasks >2014-08-11 19:20:17,549 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] FINISH, DeleteImageGroupVDSCommand, log id: 37482eaa >2014-08-11 19:20:17,668 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command c9e06dea-35fe-41d9-bb11-3f168b7ee7e2 >2014-08-11 19:20:17,668 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] CommandMultiAsyncTasks::AttachTask: Attaching task 5f4bfb54-a39f-4bd5-8fd6-ca6dc88f16a6 to command c9e06dea-35fe-41d9-bb11-3f168b7ee7e2. >2014-08-11 19:20:17,689 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] Adding task 5f4bfb54-a39f-4bd5-8fd6-ca6dc88f16a6 (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 19:20:17,713 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = cdc5a7e1-4d16-404d-90a4-c0ed6584c11b, postZeros = false, forceDelete = false), log id: 2a06ece2 >2014-08-11 19:20:18,152 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] FINISH, DeleteImageGroupVDSCommand, log id: 2a06ece2 >2014-08-11 19:20:18,161 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] CommandMultiAsyncTasks::AttachTask: Attaching task 22f78b39-5a87-4f11-b32d-d7725dda7aa9 to command c9e06dea-35fe-41d9-bb11-3f168b7ee7e2. >2014-08-11 19:20:18,177 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] Adding task 22f78b39-5a87-4f11-b32d-d7725dda7aa9 (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 19:20:18,218 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] Correlation ID: 7ae86507, Job ID: 53dfc279-bbcf-4dc9-ab63-786125de8347, Call Stack: null, Custom Event ID: -1, Message: VM vm3 powered off by admin (Host: h1) (Reason: Not Specified). >2014-08-11 19:20:18,219 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] BaseAsyncTask::startPollingTask: Starting to poll task 5f4bfb54-a39f-4bd5-8fd6-ca6dc88f16a6. >2014-08-11 19:20:18,220 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-25) [77f3ce92] BaseAsyncTask::startPollingTask: Starting to poll task 22f78b39-5a87-4f11-b32d-d7725dda7aa9. >2014-08-11 19:20:18,637 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) task id 580d0300-9d90-4abc-9558-565d9009b1d0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:18,638 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) task id ad97eeef-e100-4acb-9b01-bf8290fdcaca is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:18,639 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) task id 580d0300-9d90-4abc-9558-565d9009b1d0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:18,639 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) task id ad97eeef-e100-4acb-9b01-bf8290fdcaca is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:18,640 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-26) Polling and updating Async Tasks: 4 tasks, 2 tasks to poll now >2014-08-11 19:20:18,640 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) task id 580d0300-9d90-4abc-9558-565d9009b1d0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:18,641 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) task id ad97eeef-e100-4acb-9b01-bf8290fdcaca is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:18,657 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) task id 580d0300-9d90-4abc-9558-565d9009b1d0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:18,657 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) task id ad97eeef-e100-4acb-9b01-bf8290fdcaca is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:18,658 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) SPMAsyncTask::PollTask: Polling task 5f4bfb54-a39f-4bd5-8fd6-ca6dc88f16a6 (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 19:20:18,674 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) BaseAsyncTask::onTaskEndSuccess: Task 5f4bfb54-a39f-4bd5-8fd6-ca6dc88f16a6 (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 19:20:18,675 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-26) Task with DB Task ID 3cb8f066-6a35-48d7-be7d-4ff583b771ad and VDSM Task ID 22f78b39-5a87-4f11-b32d-d7725dda7aa9 is in state Polling. End action for command c9e06dea-35fe-41d9-bb11-3f168b7ee7e2 will proceed when all the entitys tasks are completed. >2014-08-11 19:20:18,675 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) SPMAsyncTask::PollTask: Polling task 22f78b39-5a87-4f11-b32d-d7725dda7aa9 (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 19:20:18,682 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) BaseAsyncTask::onTaskEndSuccess: Task 22f78b39-5a87-4f11-b32d-d7725dda7aa9 (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 19:20:18,683 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-26) CommandAsyncTask::endActionIfNecessary: All tasks of command c9e06dea-35fe-41d9-bb11-3f168b7ee7e2 has ended -> executing endAction >2014-08-11 19:20:18,684 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-26) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: c9e06dea-35fe-41d9-bb11-3f168b7ee7e2): calling endAction . >2014-08-11 19:20:18,684 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) task id 580d0300-9d90-4abc-9558-565d9009b1d0 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:18,684 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-48) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction StopVm, executionIndex: 0 >2014-08-11 19:20:18,685 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-26) task id ad97eeef-e100-4acb-9b01-bf8290fdcaca is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:18,699 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-48) Ending command successfully: org.ovirt.engine.core.bll.StopVmCommand >2014-08-11 19:20:18,707 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type StopVm completed, handling the result. >2014-08-11 19:20:18,707 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type StopVm succeeded, clearing tasks. >2014-08-11 19:20:18,715 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-48) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 22f78b39-5a87-4f11-b32d-d7725dda7aa9 >2014-08-11 19:20:18,716 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 22f78b39-5a87-4f11-b32d-d7725dda7aa9), log id: 4a18eda8 >2014-08-11 19:20:18,717 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=22f78b39-5a87-4f11-b32d-d7725dda7aa9), log id: 2a9ad699 >2014-08-11 19:20:18,781 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, HSMClearTaskVDSCommand, log id: 2a9ad699 >2014-08-11 19:20:18,782 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, SPMClearTaskVDSCommand, log id: 4a18eda8 >2014-08-11 19:20:18,790 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-48) BaseAsyncTask::removeTaskFromDB: Removed task 22f78b39-5a87-4f11-b32d-d7725dda7aa9 from DataBase >2014-08-11 19:20:18,790 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-48) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 5f4bfb54-a39f-4bd5-8fd6-ca6dc88f16a6 >2014-08-11 19:20:18,791 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 5f4bfb54-a39f-4bd5-8fd6-ca6dc88f16a6), log id: 22c0520f >2014-08-11 19:20:18,792 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=5f4bfb54-a39f-4bd5-8fd6-ca6dc88f16a6), log id: 4b689c >2014-08-11 19:20:18,848 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, HSMClearTaskVDSCommand, log id: 4b689c >2014-08-11 19:20:18,848 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, SPMClearTaskVDSCommand, log id: 22c0520f >2014-08-11 19:20:18,856 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-48) BaseAsyncTask::removeTaskFromDB: Removed task 5f4bfb54-a39f-4bd5-8fd6-ca6dc88f16a6 from DataBase >2014-08-11 19:20:18,856 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity c9e06dea-35fe-41d9-bb11-3f168b7ee7e2 >2014-08-11 19:20:20,803 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from Down --> Up >2014-08-11 19:20:21,978 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-35) [4aedc915] Running command: StopVmCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 19:20:21,983 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-35) [4aedc915] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@82a8f0e1), log id: 42c91dd0 >2014-08-11 19:20:21,984 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-35) [4aedc915] FINISH, UpdateVmDynamicDataVDSCommand, log id: 42c91dd0 >2014-08-11 19:20:21,996 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (org.ovirt.thread.pool-8-thread-35) [42a1c534] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-11 19:20:22,020 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-35) [42a1c534] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = ef7c8ef2-cf66-4f60-a752-f1a79235e2d6, postZeros = false, forceDelete = false), log id: 662b5e81 >2014-08-11 19:20:22,489 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-35) [42a1c534] FINISH, DeleteImageGroupVDSCommand, log id: 662b5e81 >2014-08-11 19:20:22,562 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-35) [42a1c534] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 7d54d511-50b7-479f-ab7c-8ff1922b0c00 >2014-08-11 19:20:22,562 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-35) [42a1c534] CommandMultiAsyncTasks::AttachTask: Attaching task 2f6680f9-741a-484f-966d-8dd026afd3ba to command 7d54d511-50b7-479f-ab7c-8ff1922b0c00. >2014-08-11 19:20:22,591 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-35) [42a1c534] Adding task 2f6680f9-741a-484f-966d-8dd026afd3ba (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 19:20:22,616 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-35) [42a1c534] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 4bd8a1f0-e234-40d6-a71c-9a93818794e4, postZeros = false, forceDelete = false), log id: 734232b7 >2014-08-11 19:20:23,250 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-35) [42a1c534] FINISH, DeleteImageGroupVDSCommand, log id: 734232b7 >2014-08-11 19:20:23,271 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-35) [42a1c534] CommandMultiAsyncTasks::AttachTask: Attaching task fdbc5d6e-15cb-44c4-aa3d-8c3c0b35b6f4 to command 7d54d511-50b7-479f-ab7c-8ff1922b0c00. >2014-08-11 19:20:23,287 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-35) [42a1c534] Adding task fdbc5d6e-15cb-44c4-aa3d-8c3c0b35b6f4 (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 19:20:23,328 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-35) [42a1c534] Correlation ID: 4aedc915, Job ID: 5f1ec532-f07c-4a8e-a04b-8a416f3d60e3, Call Stack: null, Custom Event ID: -1, Message: VM vm4 powered off by admin (Host: h1) (Reason: Not Specified). >2014-08-11 19:20:23,329 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-35) [42a1c534] BaseAsyncTask::startPollingTask: Starting to poll task 2f6680f9-741a-484f-966d-8dd026afd3ba. >2014-08-11 19:20:23,329 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-35) [42a1c534] BaseAsyncTask::startPollingTask: Starting to poll task fdbc5d6e-15cb-44c4-aa3d-8c3c0b35b6f4. >2014-08-11 19:20:28,687 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) task id 580d0300-9d90-4abc-9558-565d9009b1d0 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:28,687 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) task id 580d0300-9d90-4abc-9558-565d9009b1d0 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:28,688 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) task id ad97eeef-e100-4acb-9b01-bf8290fdcaca has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:28,689 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-61) Polling and updating Async Tasks: 6 tasks, 4 tasks to poll now >2014-08-11 19:20:28,689 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) task id 580d0300-9d90-4abc-9558-565d9009b1d0 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:28,690 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) task id ad97eeef-e100-4acb-9b01-bf8290fdcaca has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:28,697 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) task id 580d0300-9d90-4abc-9558-565d9009b1d0 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:28,697 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) SPMAsyncTask::PollTask: Polling task 580d0300-9d90-4abc-9558-565d9009b1d0 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 19:20:28,738 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) BaseAsyncTask::onTaskEndSuccess: Task 580d0300-9d90-4abc-9558-565d9009b1d0 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 19:20:28,738 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-61) Task with DB Task ID 99c648ea-22fe-43c7-98bf-f472031720d4 and VDSM Task ID ad97eeef-e100-4acb-9b01-bf8290fdcaca is in state Polling. End action for command a244c35d-0f2c-4d89-8296-7b7197813a17 will proceed when all the entitys tasks are completed. >2014-08-11 19:20:28,739 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) task id ad97eeef-e100-4acb-9b01-bf8290fdcaca has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-11 19:20:28,740 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) SPMAsyncTask::PollTask: Polling task ad97eeef-e100-4acb-9b01-bf8290fdcaca (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 19:20:28,779 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) BaseAsyncTask::onTaskEndSuccess: Task ad97eeef-e100-4acb-9b01-bf8290fdcaca (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 19:20:28,779 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-61) CommandAsyncTask::endActionIfNecessary: All tasks of command a244c35d-0f2c-4d89-8296-7b7197813a17 has ended -> executing endAction >2014-08-11 19:20:28,780 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-61) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: a244c35d-0f2c-4d89-8296-7b7197813a17): calling endAction . >2014-08-11 19:20:28,780 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) SPMAsyncTask::PollTask: Polling task fdbc5d6e-15cb-44c4-aa3d-8c3c0b35b6f4 (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 19:20:28,780 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-12) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RunVm, executionIndex: 0 >2014-08-11 19:20:28,787 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) BaseAsyncTask::onTaskEndSuccess: Task fdbc5d6e-15cb-44c4-aa3d-8c3c0b35b6f4 (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 19:20:28,788 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-61) Task with DB Task ID b4c47abc-2625-479b-b727-1ce6ae0b472e and VDSM Task ID 2f6680f9-741a-484f-966d-8dd026afd3ba is in state Polling. End action for command 7d54d511-50b7-479f-ab7c-8ff1922b0c00 will proceed when all the entitys tasks are completed. >2014-08-11 19:20:28,789 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) SPMAsyncTask::PollTask: Polling task 2f6680f9-741a-484f-966d-8dd026afd3ba (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 19:20:28,795 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-12) Ending command successfully: org.ovirt.engine.core.bll.RunVmCommand >2014-08-11 19:20:28,799 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-12) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm completed, handling the result. >2014-08-11 19:20:28,799 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-12) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm succeeded, clearing tasks. >2014-08-11 19:20:28,803 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) BaseAsyncTask::onTaskEndSuccess: Task 2f6680f9-741a-484f-966d-8dd026afd3ba (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 19:20:28,804 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-61) CommandAsyncTask::endActionIfNecessary: All tasks of command 7d54d511-50b7-479f-ab7c-8ff1922b0c00 has ended -> executing endAction >2014-08-11 19:20:28,804 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-61) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 7d54d511-50b7-479f-ab7c-8ff1922b0c00): calling endAction . >2014-08-11 19:20:28,805 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-37) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction StopVm, executionIndex: 0 >2014-08-11 19:20:28,812 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 580d0300-9d90-4abc-9558-565d9009b1d0 >2014-08-11 19:20:28,812 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-12) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 580d0300-9d90-4abc-9558-565d9009b1d0), log id: 782dea5c >2014-08-11 19:20:28,813 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-12) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=580d0300-9d90-4abc-9558-565d9009b1d0), log id: e757ba8 >2014-08-11 19:20:28,820 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-37) Ending command successfully: org.ovirt.engine.core.bll.StopVmCommand >2014-08-11 19:20:28,828 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-37) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type StopVm completed, handling the result. >2014-08-11 19:20:28,829 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-37) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type StopVm succeeded, clearing tasks. >2014-08-11 19:20:28,837 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-37) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 2f6680f9-741a-484f-966d-8dd026afd3ba >2014-08-11 19:20:28,838 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 2f6680f9-741a-484f-966d-8dd026afd3ba), log id: 739742d0 >2014-08-11 19:20:28,881 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-12) FINISH, HSMClearTaskVDSCommand, log id: e757ba8 >2014-08-11 19:20:28,882 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-12) FINISH, SPMClearTaskVDSCommand, log id: 782dea5c >2014-08-11 19:20:28,883 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=2f6680f9-741a-484f-966d-8dd026afd3ba), log id: 4fa3a0f9 >2014-08-11 19:20:28,895 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) BaseAsyncTask::removeTaskFromDB: Removed task 580d0300-9d90-4abc-9558-565d9009b1d0 from DataBase >2014-08-11 19:20:28,895 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) SPMAsyncTask::ClearAsyncTask: Attempting to clear task ad97eeef-e100-4acb-9b01-bf8290fdcaca >2014-08-11 19:20:28,896 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-12) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = ad97eeef-e100-4acb-9b01-bf8290fdcaca), log id: 656ff3e7 >2014-08-11 19:20:28,931 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) FINISH, HSMClearTaskVDSCommand, log id: 4fa3a0f9 >2014-08-11 19:20:28,932 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) FINISH, SPMClearTaskVDSCommand, log id: 739742d0 >2014-08-11 19:20:28,933 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-12) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=ad97eeef-e100-4acb-9b01-bf8290fdcaca), log id: 18f5f6b8 >2014-08-11 19:20:28,944 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-37) BaseAsyncTask::removeTaskFromDB: Removed task 2f6680f9-741a-484f-966d-8dd026afd3ba from DataBase >2014-08-11 19:20:28,945 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-37) SPMAsyncTask::ClearAsyncTask: Attempting to clear task fdbc5d6e-15cb-44c4-aa3d-8c3c0b35b6f4 >2014-08-11 19:20:28,946 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = fdbc5d6e-15cb-44c4-aa3d-8c3c0b35b6f4), log id: 5771b343 >2014-08-11 19:20:28,982 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-12) FINISH, HSMClearTaskVDSCommand, log id: 18f5f6b8 >2014-08-11 19:20:28,982 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-12) FINISH, SPMClearTaskVDSCommand, log id: 656ff3e7 >2014-08-11 19:20:28,983 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=fdbc5d6e-15cb-44c4-aa3d-8c3c0b35b6f4), log id: 69b0208e >2014-08-11 19:20:28,994 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-12) BaseAsyncTask::removeTaskFromDB: Removed task ad97eeef-e100-4acb-9b01-bf8290fdcaca from DataBase >2014-08-11 19:20:28,994 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-12) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity a244c35d-0f2c-4d89-8296-7b7197813a17 >2014-08-11 19:20:29,033 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) FINISH, HSMClearTaskVDSCommand, log id: 69b0208e >2014-08-11 19:20:29,033 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-37) FINISH, SPMClearTaskVDSCommand, log id: 5771b343 >2014-08-11 19:20:29,044 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-37) BaseAsyncTask::removeTaskFromDB: Removed task fdbc5d6e-15cb-44c4-aa3d-8c3c0b35b6f4 from DataBase >2014-08-11 19:20:29,044 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-37) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 7d54d511-50b7-479f-ab7c-8ff1922b0c00 >2014-08-11 19:20:29,961 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) VM vm4 de580b19-e975-426f-80de-6c7daf1f24ab moved from Down --> Up >2014-08-11 19:20:31,512 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-31) [5ab133ff] Running command: StopVmCommand internal: false. Entities affected : ID: d4f5d124-ebf2-4c76-8630-1c56803547d7 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 19:20:31,519 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-31) [5ab133ff] START, DestroyVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, force=false, secondsToWait=0, gracefully=false, reason=), log id: 40e80f9d >2014-08-11 19:20:31,523 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-31) [5ab133ff] START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=d4f5d124-ebf2-4c76-8630-1c56803547d7, force=false, secondsToWait=0, gracefully=false, reason=), log id: 58656179 >2014-08-11 19:20:31,781 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-31) [5ab133ff] FINISH, DestroyVDSCommand, log id: 58656179 >2014-08-11 19:20:31,810 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-31) [5ab133ff] FINISH, DestroyVmVDSCommand, return: Down, log id: 40e80f9d >2014-08-11 19:20:31,813 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-22) [26145990] Running command: ProcessDownVmCommand internal: true. >2014-08-11 19:20:31,818 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-31) [5ab133ff] Correlation ID: 5ab133ff, Job ID: f63367a6-b025-4ea9-9715-fd6fd10fa580, Call Stack: null, Custom Event ID: -1, Message: VM vm5 powered off by admin (Host: h1) (Reason: Not Specified). >2014-08-11 19:20:38,311 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-28) [414ea5a0] Running command: StopVmCommand internal: false. Entities affected : ID: 32f54e3c-df93-4be3-95c2-b843604932f9 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 19:20:38,315 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-28) [414ea5a0] START, DestroyVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=32f54e3c-df93-4be3-95c2-b843604932f9, force=false, secondsToWait=0, gracefully=false, reason=), log id: 53de3f9f >2014-08-11 19:20:38,317 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-28) [414ea5a0] START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=32f54e3c-df93-4be3-95c2-b843604932f9, force=false, secondsToWait=0, gracefully=false, reason=), log id: 2c17b58c >2014-08-11 19:20:38,574 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-28) [414ea5a0] FINISH, DestroyVDSCommand, log id: 2c17b58c >2014-08-11 19:20:38,585 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-28) [414ea5a0] FINISH, DestroyVmVDSCommand, return: Down, log id: 53de3f9f >2014-08-11 19:20:38,588 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-17) [603d13eb] Running command: ProcessDownVmCommand internal: true. >2014-08-11 19:20:38,593 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-28) [414ea5a0] Correlation ID: 414ea5a0, Job ID: a01a599c-0d71-4af9-b30b-cf422bd11b8a, Call Stack: null, Custom Event ID: -1, Message: VM vm6 powered off by admin (Host: h1) (Reason: Not Specified). >2014-08-11 19:20:47,969 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-25) [32bc87f6] Running command: StopVmCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 19:20:47,972 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-25) [32bc87f6] START, DestroyVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, force=false, secondsToWait=0, gracefully=false, reason=), log id: 32faa018 >2014-08-11 19:20:47,974 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-25) [32bc87f6] START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, force=false, secondsToWait=0, gracefully=false, reason=), log id: 26e8dae2 >2014-08-11 19:20:48,274 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-25) [32bc87f6] FINISH, DestroyVDSCommand, log id: 26e8dae2 >2014-08-11 19:20:48,284 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-25) [32bc87f6] FINISH, DestroyVmVDSCommand, return: Down, log id: 32faa018 >2014-08-11 19:20:48,287 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-43) [20fdc7d8] Running command: ProcessDownVmCommand internal: true. >2014-08-11 19:20:48,300 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-25) [32bc87f6] Correlation ID: 32bc87f6, Job ID: b79d213d-6c6d-468b-a66a-3d9d073b14df, Call Stack: null, Custom Event ID: -1, Message: VM vm4 powered off by admin (Host: h1) (Reason: Not Specified). >2014-08-11 19:21:00,246 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-49) [55fa76fe] Running command: StopVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-11 19:21:00,250 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-49) [55fa76fe] START, DestroyVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, force=false, secondsToWait=0, gracefully=false, reason=), log id: 1d9a90b1 >2014-08-11 19:21:00,254 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-49) [55fa76fe] START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, force=false, secondsToWait=0, gracefully=false, reason=), log id: 65cfeac9 >2014-08-11 19:21:00,516 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-49) [55fa76fe] FINISH, DestroyVDSCommand, log id: 65cfeac9 >2014-08-11 19:21:00,528 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-49) [55fa76fe] FINISH, DestroyVmVDSCommand, return: Down, log id: 1d9a90b1 >2014-08-11 19:21:00,536 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-49) [55fa76fe] Correlation ID: 55fa76fe, Job ID: dfab51ff-c869-46f6-bd2e-fb7a251fa752, Call Stack: null, Custom Event ID: -1, Message: VM vm3 powered off by admin (Host: h1) (Reason: Not Specified). >2014-08-11 19:21:00,552 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-15) [1db338c9] Running command: ProcessDownVmCommand internal: true. >2014-08-11 19:21:47,494 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-92) Setting new tasks map. The map contains now 0 tasks >2014-08-11 19:21:47,495 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-92) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-11 19:28:38,789 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-45) Principal admin::internal synchronized >2014-08-11 19:28:43,650 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VMs/Templates Ovf. >2014-08-11 19:28:43,838 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-40) [37c3ca54] Lock Acquired to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-11 19:28:43,841 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-40) [37c3ca54] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-11 19:28:43,851 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [37c3ca54] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 32fca619 >2014-08-11 19:28:43,853 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [37c3ca54] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-11 19:28:43,854 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [37c3ca54] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Mon Aug 11 16:28:39 IDT 2014"} > >2014-08-11 19:28:43,967 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [37c3ca54] FINISH, SetVolumeDescriptionVDSCommand, log id: 32fca619 >2014-08-11 19:28:44,026 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-11 19:28:44,027 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 7a1dfef3 >2014-08-11 19:28:44,027 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] -- executeVdsBrokerCommand: >2014-08-11 19:28:44,028 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > size=71,680 > >2014-08-11 19:28:46,028 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] FINISH, UploadStreamVDSCommand, log id: 7a1dfef3 >2014-08-11 19:28:46,047 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-40) [3111a6] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 2da88df8-ef07-441e-a835-b0ab208f09cd >2014-08-11 19:28:46,048 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-40) [3111a6] CommandMultiAsyncTasks::AttachTask: Attaching task 9e1810ef-ed42-4093-b638-2a6323bd216d to command 2da88df8-ef07-441e-a835-b0ab208f09cd. >2014-08-11 19:28:46,064 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-40) [3111a6] Adding task 9e1810ef-ed42-4093-b638-2a6323bd216d (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 19:28:46,080 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-11 19:28:46,085 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 75798347 >2014-08-11 19:28:46,086 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-11 19:28:46,086 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Mon Aug 11 19:28:43 IDT 2014","Size":71680} > >2014-08-11 19:28:46,192 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] FINISH, SetVolumeDescriptionVDSCommand, log id: 75798347 >2014-08-11 19:28:46,206 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 152b1620 >2014-08-11 19:28:46,207 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-11 19:28:46,208 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Mon Aug 11 16:28:39 IDT 2014"} > >2014-08-11 19:28:46,327 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [3111a6] FINISH, SetVolumeDescriptionVDSCommand, log id: 152b1620 >2014-08-11 19:28:46,412 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-40) [2a97561c] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-11 19:28:46,413 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-40) [2a97561c] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: d49d495 >2014-08-11 19:28:46,414 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-40) [2a97561c] -- executeVdsBrokerCommand: >2014-08-11 19:28:46,414 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-40) [2a97561c] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > size=71,680 > >2014-08-11 19:28:48,229 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-40) [2a97561c] FINISH, UploadStreamVDSCommand, log id: d49d495 >2014-08-11 19:28:48,284 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-40) [2a97561c] CommandMultiAsyncTasks::AttachTask: Attaching task 0193a5ee-7830-4b3f-8076-a924b716ab9c to command 2da88df8-ef07-441e-a835-b0ab208f09cd. >2014-08-11 19:28:48,300 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-40) [2a97561c] Adding task 0193a5ee-7830-4b3f-8076-a924b716ab9c (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-11 19:28:48,316 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-40) [2a97561c] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-11 19:28:48,317 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-40) [2a97561c] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-11 19:28:48,318 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-40) [2a97561c] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-11 19:28:48,319 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-40) [2a97561c] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-11 19:28:48,319 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-40) [2a97561c] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-11 19:28:48,322 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [2a97561c] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 626c9054 >2014-08-11 19:28:48,323 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [2a97561c] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-11 19:28:48,324 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [2a97561c] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Mon Aug 11 19:28:43 IDT 2014","Size":71680} > >2014-08-11 19:28:48,413 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-40) [2a97561c] FINISH, SetVolumeDescriptionVDSCommand, log id: 626c9054 >2014-08-11 19:28:48,424 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-40) [2a97561c] BaseAsyncTask::startPollingTask: Starting to poll task 9e1810ef-ed42-4093-b638-2a6323bd216d. >2014-08-11 19:28:48,424 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-40) [2a97561c] BaseAsyncTask::startPollingTask: Starting to poll task 0193a5ee-7830-4b3f-8076-a924b716ab9c. >2014-08-11 19:28:48,815 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-61) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-11 19:28:48,838 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) SPMAsyncTask::PollTask: Polling task 9e1810ef-ed42-4093-b638-2a6323bd216d (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 19:28:48,839 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) BaseAsyncTask::onTaskEndSuccess: Task 9e1810ef-ed42-4093-b638-2a6323bd216d (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 19:28:48,840 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-61) Task with DB Task ID d6265777-b08b-44c2-b549-128071f56d0a and VDSM Task ID 0193a5ee-7830-4b3f-8076-a924b716ab9c is in state Polling. End action for command 2da88df8-ef07-441e-a835-b0ab208f09cd will proceed when all the entitys tasks are completed. >2014-08-11 19:28:48,841 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) SPMAsyncTask::PollTask: Polling task 0193a5ee-7830-4b3f-8076-a924b716ab9c (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-11 19:28:48,842 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-61) BaseAsyncTask::onTaskEndSuccess: Task 0193a5ee-7830-4b3f-8076-a924b716ab9c (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-11 19:28:48,844 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-61) CommandAsyncTask::endActionIfNecessary: All tasks of command 2da88df8-ef07-441e-a835-b0ab208f09cd has ended -> executing endAction >2014-08-11 19:28:48,845 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-61) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 2da88df8-ef07-441e-a835-b0ab208f09cd): calling endAction . >2014-08-11 19:28:48,845 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-08-11 19:28:48,865 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-08-11 19:28:48,865 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-11 19:28:48,866 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-11 19:28:48,867 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-11 19:28:48,867 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-11 19:28:48,868 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-11 19:28:48,869 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-08-11 19:28:48,870 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-08-11 19:28:48,870 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 9e1810ef-ed42-4093-b638-2a6323bd216d >2014-08-11 19:28:48,871 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 9e1810ef-ed42-4093-b638-2a6323bd216d), log id: 6d3d8767 >2014-08-11 19:28:48,872 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=9e1810ef-ed42-4093-b638-2a6323bd216d), log id: 6284b338 >2014-08-11 19:28:48,982 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] FINISH, HSMClearTaskVDSCommand, log id: 6284b338 >2014-08-11 19:28:48,983 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] FINISH, SPMClearTaskVDSCommand, log id: 6d3d8767 >2014-08-11 19:28:49,005 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] BaseAsyncTask::removeTaskFromDB: Removed task 9e1810ef-ed42-4093-b638-2a6323bd216d from DataBase >2014-08-11 19:28:49,006 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 0193a5ee-7830-4b3f-8076-a924b716ab9c >2014-08-11 19:28:49,007 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 0193a5ee-7830-4b3f-8076-a924b716ab9c), log id: 18324126 >2014-08-11 19:28:49,009 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=0193a5ee-7830-4b3f-8076-a924b716ab9c), log id: b533233 >2014-08-11 19:28:49,062 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] FINISH, HSMClearTaskVDSCommand, log id: b533233 >2014-08-11 19:28:49,063 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] FINISH, SPMClearTaskVDSCommand, log id: 18324126 >2014-08-11 19:28:49,088 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] BaseAsyncTask::removeTaskFromDB: Removed task 0193a5ee-7830-4b3f-8076-a924b716ab9c from DataBase >2014-08-11 19:28:49,089 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-49) [37c3ca54] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 2da88df8-ef07-441e-a835-b0ab208f09cd >2014-08-11 19:30:17,501 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-25) Setting new tasks map. The map contains now 0 tasks >2014-08-11 19:30:17,501 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-25) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-12 07:25:52,673 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-2) Start deleteAgedOutAuditLogs >2014-08-12 07:25:52,675 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-28) Attempting to update VMs/Templates Ovf. >2014-08-12 07:25:52,676 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-3) Start deleteAgedOutCommandEntities >2014-08-12 07:25:53,375 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-11) Principal admin::internal synchronized >2014-08-12 07:25:53,377 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-3) Finished deleteAgedOutCommandEntities >2014-08-12 07:25:53,387 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-2) Finished deleteAgedOutAuditLogs >2014-08-12 07:25:53,399 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-66) Command SpmStatusVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-12 07:25:53,407 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-66) [716fda8f] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-12 07:25:53,448 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-66) [716fda8f] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-12 07:25:53,449 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-66) [716fda8f] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-12 07:25:53,618 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-66) [716fda8f] Correlation ID: 716fda8f, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host h1, Error: Network error during communication with the Host.). >2014-08-12 07:26:06,438 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) [716fda8f] IrsBroker::Failed::GetStoragePoolInfoVDS due to: SocketException: Connection reset >2014-08-12 07:26:07,667 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetStatsVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketException: Connection reset >2014-08-12 07:26:07,671 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) Host h1 is not responding. It will stay in Connecting state for a grace period of $81 seconds and after that an attempt to fence the host will be issued. >2014-08-12 07:26:07,702 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) vds::refreshVdsStats Failed getVdsStats, vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, error = VDSNetworkException: java.net.SocketException: Connection reset >2014-08-12 07:26:07,703 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketException: Connection reset > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsStats(VdsUpdateRunTimeInfo.java:669) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:623) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor55.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketException: Connection reset > at java.net.SocketInputStream.read(SocketInputStream.java:189) [rt.jar:1.7.0_25] > at java.net.SocketInputStream.read(SocketInputStream.java:121) [rt.jar:1.7.0_25] > at sun.security.ssl.InputRecord.readFully(InputRecord.java:442) [jsse.jar:1.7.0_25] > at sun.security.ssl.InputRecord.read(InputRecord.java:480) [jsse.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927) [jsse.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884) [jsse.jar:1.7.0_25] > at sun.security.ssl.AppInputStream.read(AppInputStream.java:102) [jsse.jar:1.7.0_25] > at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) [rt.jar:1.7.0_25] > at java.io.BufferedInputStream.read(BufferedInputStream.java:254) [rt.jar:1.7.0_25] > at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1413) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsStats(Unknown Source) > at sun.reflect.GeneratedMethodAccessor76.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-12 07:26:07,833 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.net.SocketException: Connection reset >2014-08-12 07:26:11,206 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-40) [2a97561c] START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 42c3a8d7 >2014-08-12 07:26:11,213 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-40) [2a97561c] FINISH, GetHardwareInfoVDSCommand, log id: 42c3a8d7 >2014-08-12 07:26:11,284 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-40) [2a97561c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-12 07:26:11,285 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) [2a97561c] Host h1 is running with disabled SELinux. >2014-08-12 07:26:11,336 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-2) Running command: LoginUserCommand internal: false. >2014-08-12 07:26:11,362 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-40) [31da6867] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-12 07:26:11,367 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-12 07:26:11,379 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-40) [2245a405] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-12 07:26:11,454 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-40) [4bfbfec7] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-12 07:26:11,481 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-40) [4bfbfec7] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = NFS, connectionList = [{ id: 1670ace3-7db6-4c73-9b3c-614dcc85bd46, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc5, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc4, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 427165c9 >2014-08-12 07:26:11,505 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-40) [4bfbfec7] FINISH, ConnectStorageServerVDSCommand, return: {1670ace3-7db6-4c73-9b3c-614dcc85bd46=0, 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a=0}, log id: 427165c9 >2014-08-12 07:26:11,506 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-40) [4bfbfec7] Host h1 storage connection was succeeded >2014-08-12 07:26:11,517 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-12) START, ConnectStoragePoolVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vdsId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, masterVersion = 1), log id: 22328855 >2014-08-12 07:26:11,936 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-12) FINISH, ConnectStoragePoolVDSCommand, log id: 22328855 >2014-08-12 07:26:11,961 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-40) [4bfbfec7] START, SetMOMPolicyParametersVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 2b86440f >2014-08-12 07:26:12,013 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-40) [4bfbfec7] FINISH, SetMOMPolicyParametersVDSCommand, log id: 2b86440f >2014-08-12 07:26:12,038 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-40) [4bfbfec7] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host h1 was set to Up. >2014-08-12 07:26:12,045 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-40) [4bfbfec7] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h1. >2014-08-12 07:26:12,067 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-40) [1411c158] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-12 07:26:16,982 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 51 GB of free space >2014-08-12 07:26:17,007 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 51 GB of free space >2014-08-12 07:26:27,098 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-2) [1e1558ad] Running command: LogoutUserCommand internal: false. >2014-08-12 07:26:27,103 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-5) [56a23d89] Running command: LogoutUserCommand internal: false. >2014-08-12 07:26:27,107 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-7) [3c365fd7] Running command: LogoutUserCommand internal: false. >2014-08-12 07:26:27,135 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [1e1558ad] Correlation ID: 1e1558ad, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-12 07:26:27,139 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-13) [550c3345] Running command: LogoutUserCommand internal: false. >2014-08-12 07:26:27,151 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-1) [11dfdd12] Running command: LogoutUserCommand internal: false. >2014-08-12 07:26:27,154 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-9) [3806ddc] Running command: LogoutUserCommand internal: false. >2014-08-12 07:26:27,170 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-7) [3c365fd7] Correlation ID: 3c365fd7, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-12 07:26:27,177 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-5) [56a23d89] Correlation ID: 56a23d89, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-12 07:26:27,177 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-2) [645bbb4e] Running command: LogoutUserCommand internal: false. >2014-08-12 07:26:27,182 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [11dfdd12] Correlation ID: 11dfdd12, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-12 07:26:27,187 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-14) [16296a4f] Running command: LogoutUserCommand internal: false. >2014-08-12 07:26:27,189 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-7) [696641e8] Running command: LogoutUserCommand internal: false. >2014-08-12 07:26:27,190 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-13) [550c3345] Correlation ID: 550c3345, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-12 07:26:27,198 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-9) [3806ddc] Correlation ID: 3806ddc, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-12 07:26:27,210 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-14) [16296a4f] Correlation ID: 16296a4f, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-12 07:26:27,212 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [645bbb4e] Correlation ID: 645bbb4e, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-12 07:26:27,227 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-7) [696641e8] Correlation ID: 696641e8, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-12 08:25:53,445 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-41) [3322697f] Attempting to update VMs/Templates Ovf. >2014-08-12 08:25:53,463 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-82) Principal admin::internal synchronized >2014-08-12 09:25:53,484 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-51) Principal admin::internal synchronized >2014-08-12 09:25:53,488 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-24) Attempting to update VMs/Templates Ovf. >2014-08-12 10:25:53,515 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-82) Attempting to update VMs/Templates Ovf. >2014-08-12 10:25:53,541 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-97) Principal admin::internal synchronized >2014-08-12 11:25:53,540 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-34) Attempting to update VMs/Templates Ovf. >2014-08-12 11:25:53,602 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-31) Principal admin::internal synchronized >2014-08-12 12:25:53,583 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VMs/Templates Ovf. >2014-08-12 12:25:53,614 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-81) Principal admin::internal synchronized >2014-08-12 13:45:26,732 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-12 13:45:26,774 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-12 13:45:26,776 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-12 13:45:26,777 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-12 13:45:26,778 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-12 13:45:26,779 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-12 13:45:26,780 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-12 13:45:26,780 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-12 13:45:26,781 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-12 13:45:26,781 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-12 13:45:26,782 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-12 13:45:26,782 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-12 13:45:26,783 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-12 13:45:26,784 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-12 13:45:26,784 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-12 13:45:26,785 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-12 13:45:26,785 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-12 13:45:26,786 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-12 13:45:26,786 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-12 13:45:26,787 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-12 13:45:26,787 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-12 13:45:26,788 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-12 13:45:26,788 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_USER" is "engine". >2014-08-12 13:45:26,789 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-12 13:45:26,789 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-12 13:45:26,790 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-12 13:45:26,791 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-12 13:45:26,792 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-12 13:45:26,793 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-12 13:45:26,793 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-12 13:45:26,793 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-12 13:45:26,794 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-12 13:45:26,794 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-12 13:45:26,795 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-12 13:45:26,795 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-12 13:45:26,796 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-12 13:45:26,796 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-12 13:45:26,797 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-12 13:45:26,798 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-12 13:45:26,798 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-12 13:45:26,799 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-12 13:45:26,799 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-12 13:45:26,800 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-12 13:45:26,800 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-12 13:45:26,801 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-12 13:45:26,802 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-12 13:45:26,803 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-12 13:45:26,803 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-12 13:45:26,804 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-12 13:45:26,804 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-12 13:45:26,805 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-12 13:45:26,805 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-12 13:45:26,806 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-12 13:45:26,807 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-12 13:45:26,807 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-12 13:45:26,808 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-12 13:45:26,808 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-12 13:45:26,809 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-12 13:45:26,809 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-12 13:45:26,810 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-12 13:45:26,810 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-12 13:45:26,811 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-12 13:45:26,811 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-12 13:45:26,812 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_USER" is "ovirt". >2014-08-12 13:45:26,812 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-12 13:45:26,813 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-12 13:45:26,814 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-12 13:45:26,814 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-12 13:45:26,815 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-12 13:45:26,824 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-12 13:45:27,184 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Start initializing Backend >2014-08-12 13:45:27,357 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-12 13:45:27,462 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-12 13:45:27,508 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-12 13:45:27,529 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-1) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-12 13:45:27,688 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Running ovirt-engine 3.5.0.0 >2014-08-12 13:45:27,688 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Start initializing dictionaries >2014-08-12 13:45:27,691 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Finished initializing dictionaries >2014-08-12 13:45:27,692 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing AuditLogCleanupManager >2014-08-12 13:45:27,693 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-12 13:45:27,710 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing AuditLogCleanupManager >2014-08-12 13:45:27,710 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing CommandEntityCleanupManager >2014-08-12 13:45:27,710 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-12 13:45:27,711 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing CommandEntityCleanupManager >2014-08-12 13:45:27,713 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Start initializing TagsDirector >2014-08-12 13:45:27,723 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Tag root added to tree >2014-08-12 13:45:27,731 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Finished initializing TagsDirector >2014-08-12 13:45:27,732 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Start initializing IsoDomainListSyncronizer >2014-08-12 13:45:27,738 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Finished initializing IsoDomainListSyncronizer >2014-08-12 13:45:27,805 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Completed initializing handlers >2014-08-12 13:45:27,823 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-08-12 13:45:27,833 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-12 13:45:27,834 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-08-12 13:45:27,834 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-08-12 13:45:27,835 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-08-12 13:45:27,836 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Mark incomplete jobs as UNKNOWN >2014-08-12 13:45:27,894 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Start initializing JobRepositoryCleanupManager >2014-08-12 13:45:27,895 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Finished initializing JobRepositoryCleanupManager >2014-08-12 13:45:27,896 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Start initializing AutoRecoveryManager >2014-08-12 13:45:27,897 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Finished initializing AutoRecoveryManager >2014-08-12 13:45:27,899 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Start initializing ExecutionMessageDirector >2014-08-12 13:45:27,900 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Finished initializing ExecutionMessageDirector >2014-08-12 13:45:27,963 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Start of enabled extensions list >2014-08-12 13:45:27,964 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-12 13:45:27,964 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-12 13:45:27,965 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) End of enabled extensions list >2014-08-12 13:45:28,035 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-8) Start initializing DbUserCacheManager >2014-08-12 13:45:28,036 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-8) Finished initializing DbUserCacheManager >2014-08-12 13:45:28,043 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-8) Initialization of AsyncTaskManager completed successfully. >2014-08-12 13:45:28,045 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-8) Start initializing ResourceManager >2014-08-12 13:45:28,077 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-12 13:45:28,104 ERROR [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-8) Failed to initialize backend: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [select * from getallfromvms(?, ?)]; nested exception is org.postgresql.util.PSQLException: The column name vm_mem_size_mb was not found in this ResultSet. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:98) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VmDAODbFacadeImpl.getAll(VmDAODbFacadeImpl.java:216) [dal.jar:] > at org.ovirt.engine.core.dao.VmDAODbFacadeImpl.getAll(VmDAODbFacadeImpl.java:211) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.init(ResourceManager.java:82) [vdsbroker.jar:] > at org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean.create(InitBackendServicesOnStartupBean.java:66) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] >Caused by: org.postgresql.util.PSQLException: The column name vm_mem_size_mb was not found in this ResultSet. > at org.postgresql.jdbc2.AbstractJdbc2ResultSet.findColumn(AbstractJdbc2ResultSet.java:2542) > at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getInt(AbstractJdbc2ResultSet.java:2406) > at org.jboss.jca.adapters.jdbc.WrappedResultSet.getInt(WrappedResultSet.java:1073) > at org.ovirt.engine.core.dao.VmDAODbFacadeImpl$VMRowMapper.mapRow(VmDAODbFacadeImpl.java:317) [dal.jar:] > at org.ovirt.engine.core.dao.VmDAODbFacadeImpl$VMRowMapper.mapRow(VmDAODbFacadeImpl.java:305) [dal.jar:] > at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:92) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:1) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:649) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 47 more > >2014-08-12 13:47:40,453 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-12 13:47:40,455 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-12 13:47:40,459 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-12 13:47:40,460 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-12 13:47:40,461 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-12 13:47:40,465 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-12 13:47:40,467 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-12 13:47:40,468 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-12 13:47:40,468 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-12 13:47:40,469 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-12 13:47:40,470 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-12 13:47:40,470 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-12 13:47:40,471 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-12 13:47:40,472 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-12 13:47:40,473 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-12 13:47:40,474 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-12 13:47:40,474 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-12 13:47:40,475 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-12 13:47:40,476 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-12 13:47:40,476 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-12 13:47:40,477 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-12 13:47:40,477 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-12 13:47:40,478 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_USER" is "engine". >2014-08-12 13:47:40,478 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-12 13:47:40,479 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-12 13:47:40,479 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-12 13:47:40,480 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-12 13:47:40,481 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-12 13:47:40,481 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-12 13:47:40,482 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-12 13:47:40,482 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-12 13:47:40,482 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-12 13:47:40,483 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-12 13:47:40,483 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-12 13:47:40,484 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-12 13:47:40,484 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-12 13:47:40,485 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-12 13:47:40,486 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-12 13:47:40,486 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-12 13:47:40,487 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-12 13:47:40,487 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-12 13:47:40,488 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-12 13:47:40,489 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-12 13:47:40,489 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-12 13:47:40,490 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-12 13:47:40,490 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-12 13:47:40,491 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-12 13:47:40,491 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-12 13:47:40,492 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-12 13:47:40,492 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-12 13:47:40,493 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-12 13:47:40,493 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-12 13:47:40,494 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-12 13:47:40,495 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-12 13:47:40,495 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-12 13:47:40,496 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-12 13:47:40,496 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-12 13:47:40,497 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-12 13:47:40,497 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-12 13:47:40,498 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-12 13:47:40,498 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-12 13:47:40,499 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-12 13:47:40,499 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-12 13:47:40,500 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_USER" is "ovirt". >2014-08-12 13:47:40,500 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-12 13:47:40,501 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-12 13:47:40,502 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-12 13:47:40,502 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-12 13:47:40,503 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-12 13:47:40,504 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-12 13:47:40,761 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-4) Start initializing Backend >2014-08-12 13:47:40,879 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-4) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-12 13:47:40,973 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-4) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-12 13:47:41,010 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-4) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-12 13:47:41,022 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-4) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-12 13:47:41,594 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-4) Running ovirt-engine 3.5.0.0 >2014-08-12 13:47:41,595 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-4) Start initializing dictionaries >2014-08-12 13:47:41,598 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-4) Finished initializing dictionaries >2014-08-12 13:47:41,599 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-4) Start initializing AuditLogCleanupManager >2014-08-12 13:47:41,599 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-4) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-12 13:47:41,617 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-4) Finished initializing AuditLogCleanupManager >2014-08-12 13:47:41,617 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-4) Start initializing CommandEntityCleanupManager >2014-08-12 13:47:41,618 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-4) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-12 13:47:41,619 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-4) Finished initializing CommandEntityCleanupManager >2014-08-12 13:47:41,621 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-4) Start initializing TagsDirector >2014-08-12 13:47:41,635 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-4) Tag root added to tree >2014-08-12 13:47:41,643 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-4) Finished initializing TagsDirector >2014-08-12 13:47:41,644 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-4) Start initializing IsoDomainListSyncronizer >2014-08-12 13:47:41,648 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-4) Finished initializing IsoDomainListSyncronizer >2014-08-12 13:47:41,712 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-4) Completed initializing handlers >2014-08-12 13:47:41,730 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-4) Start initializing ErrorTranslatorImpl >2014-08-12 13:47:41,738 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-4) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-12 13:47:41,740 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-4) Finished initializing ErrorTranslatorImpl >2014-08-12 13:47:41,740 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-4) Start initializing ErrorTranslatorImpl >2014-08-12 13:47:41,741 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-4) Finished initializing ErrorTranslatorImpl >2014-08-12 13:47:41,741 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-4) Mark incomplete jobs as UNKNOWN >2014-08-12 13:47:41,762 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-4) Start initializing JobRepositoryCleanupManager >2014-08-12 13:47:41,762 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-4) Finished initializing JobRepositoryCleanupManager >2014-08-12 13:47:41,763 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-4) Start initializing AutoRecoveryManager >2014-08-12 13:47:41,764 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-4) Finished initializing AutoRecoveryManager >2014-08-12 13:47:41,765 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-4) Start initializing ExecutionMessageDirector >2014-08-12 13:47:41,767 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-4) Finished initializing ExecutionMessageDirector >2014-08-12 13:47:41,811 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-4) Start of enabled extensions list >2014-08-12 13:47:41,812 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-4) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-12 13:47:41,812 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-4) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-12 13:47:41,813 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-4) End of enabled extensions list >2014-08-12 13:47:41,880 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-5) Start initializing DbUserCacheManager >2014-08-12 13:47:41,882 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-5) Finished initializing DbUserCacheManager >2014-08-12 13:47:41,891 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-5) Initialization of AsyncTaskManager completed successfully. >2014-08-12 13:47:41,892 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-5) Start initializing ResourceManager >2014-08-12 13:47:41,906 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-12 13:47:41,942 ERROR [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-5) Failed to initialize backend: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [select * from getallfromvms(?, ?)]; nested exception is org.postgresql.util.PSQLException: The column name vm_mem_size_mb was not found in this ResultSet. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:98) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VmDAODbFacadeImpl.getAll(VmDAODbFacadeImpl.java:216) [dal.jar:] > at org.ovirt.engine.core.dao.VmDAODbFacadeImpl.getAll(VmDAODbFacadeImpl.java:211) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.init(ResourceManager.java:82) [vdsbroker.jar:] > at org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean.create(InitBackendServicesOnStartupBean.java:66) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] >Caused by: org.postgresql.util.PSQLException: The column name vm_mem_size_mb was not found in this ResultSet. > at org.postgresql.jdbc2.AbstractJdbc2ResultSet.findColumn(AbstractJdbc2ResultSet.java:2542) > at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getInt(AbstractJdbc2ResultSet.java:2406) > at org.jboss.jca.adapters.jdbc.WrappedResultSet.getInt(WrappedResultSet.java:1073) > at org.ovirt.engine.core.dao.VmDAODbFacadeImpl$VMRowMapper.mapRow(VmDAODbFacadeImpl.java:317) [dal.jar:] > at org.ovirt.engine.core.dao.VmDAODbFacadeImpl$VMRowMapper.mapRow(VmDAODbFacadeImpl.java:305) [dal.jar:] > at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:92) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:1) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:649) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 47 more > >2014-08-12 13:58:57,262 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-12 13:58:57,263 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-12 13:58:57,281 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-12 13:58:57,282 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-12 13:58:57,283 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-12 13:58:57,284 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-12 13:58:57,285 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-12 13:58:57,286 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-12 13:58:57,286 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-12 13:58:57,287 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-12 13:58:57,287 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-12 13:58:57,288 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-12 13:58:57,288 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-12 13:58:57,289 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-12 13:58:57,289 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-12 13:58:57,290 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-12 13:58:57,290 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-12 13:58:57,291 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-12 13:58:57,291 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-12 13:58:57,292 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-12 13:58:57,292 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-12 13:58:57,293 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-12 13:58:57,294 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_USER" is "engine". >2014-08-12 13:58:57,294 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-12 13:58:57,295 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-12 13:58:57,295 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-12 13:58:57,296 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-12 13:58:57,297 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-12 13:58:57,297 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-12 13:58:57,298 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-12 13:58:57,299 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-12 13:58:57,299 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-12 13:58:57,300 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-12 13:58:57,300 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-12 13:58:57,301 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-12 13:58:57,301 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-12 13:58:57,302 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-12 13:58:57,303 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-12 13:58:57,303 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-12 13:58:57,304 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-12 13:58:57,305 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-12 13:58:57,306 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-12 13:58:57,307 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-12 13:58:57,307 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-12 13:58:57,308 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-12 13:58:57,308 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-12 13:58:57,309 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-12 13:58:57,310 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-12 13:58:57,311 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-12 13:58:57,311 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-12 13:58:57,312 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-12 13:58:57,313 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-12 13:58:57,313 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-12 13:58:57,314 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-12 13:58:57,314 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-12 13:58:57,315 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-12 13:58:57,316 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-12 13:58:57,316 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-12 13:58:57,317 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-12 13:58:57,317 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-12 13:58:57,318 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-12 13:58:57,318 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-12 13:58:57,319 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-12 13:58:57,319 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_USER" is "ovirt". >2014-08-12 13:58:57,320 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-12 13:58:57,320 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-12 13:58:57,321 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-12 13:58:57,322 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-12 13:58:57,322 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-12 13:58:57,323 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-12 13:58:57,864 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Start initializing Backend >2014-08-12 13:58:58,042 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-7) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-12 13:58:58,150 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-7) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-12 13:58:58,190 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-7) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-12 13:58:58,201 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-7) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-12 13:58:58,444 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Running ovirt-engine 3.5.0.0 >2014-08-12 13:58:58,445 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-7) Start initializing dictionaries >2014-08-12 13:58:58,448 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-7) Finished initializing dictionaries >2014-08-12 13:58:58,448 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Start initializing AuditLogCleanupManager >2014-08-12 13:58:58,449 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-12 13:58:58,468 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Finished initializing AuditLogCleanupManager >2014-08-12 13:58:58,468 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Start initializing CommandEntityCleanupManager >2014-08-12 13:58:58,469 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-12 13:58:58,470 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Finished initializing CommandEntityCleanupManager >2014-08-12 13:58:58,471 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-7) Start initializing TagsDirector >2014-08-12 13:58:58,482 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-7) Tag root added to tree >2014-08-12 13:58:58,491 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-7) Finished initializing TagsDirector >2014-08-12 13:58:58,492 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-7) Start initializing IsoDomainListSyncronizer >2014-08-12 13:58:58,498 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-7) Finished initializing IsoDomainListSyncronizer >2014-08-12 13:58:58,571 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Completed initializing handlers >2014-08-12 13:58:58,590 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Start initializing ErrorTranslatorImpl >2014-08-12 13:58:58,600 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-12 13:58:58,602 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Finished initializing ErrorTranslatorImpl >2014-08-12 13:58:58,602 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Start initializing ErrorTranslatorImpl >2014-08-12 13:58:58,603 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Finished initializing ErrorTranslatorImpl >2014-08-12 13:58:58,603 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Mark incomplete jobs as UNKNOWN >2014-08-12 13:58:58,627 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-7) Start initializing JobRepositoryCleanupManager >2014-08-12 13:58:58,628 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-7) Finished initializing JobRepositoryCleanupManager >2014-08-12 13:58:58,628 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-7) Start initializing AutoRecoveryManager >2014-08-12 13:58:58,629 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-7) Finished initializing AutoRecoveryManager >2014-08-12 13:58:58,631 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-7) Start initializing ExecutionMessageDirector >2014-08-12 13:58:58,632 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-7) Finished initializing ExecutionMessageDirector >2014-08-12 13:58:58,672 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-7) Start of enabled extensions list >2014-08-12 13:58:58,673 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-7) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-12 13:58:58,673 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-7) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-12 13:58:58,674 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-7) End of enabled extensions list >2014-08-12 13:58:58,737 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-3) Start initializing DbUserCacheManager >2014-08-12 13:58:58,738 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-3) Finished initializing DbUserCacheManager >2014-08-12 13:58:58,754 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-3) Initialization of AsyncTaskManager completed successfully. >2014-08-12 13:58:58,757 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-3) Start initializing ResourceManager >2014-08-12 13:58:58,763 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-12 13:58:58,841 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-3) Entered VdsManager constructor >2014-08-12 13:58:58,863 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-3) Initialize vdsBroker (galactica,54,321) >2014-08-12 13:58:58,930 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-3) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-12 13:58:58,931 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-3) Entered VdsManager constructor >2014-08-12 13:58:58,938 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-3) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-12 13:58:59,009 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-3) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-12 13:58:59,027 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-3) Finished initializing ResourceManager >2014-08-12 13:58:59,032 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-3) Initialization of OvfDataUpdater completed successfully. >2014-08-12 13:58:59,033 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) Start scheduling to enable vds load balancer >2014-08-12 13:58:59,035 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) Finished scheduling to enable vds load balancer >2014-08-12 13:58:59,035 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) Start HA Reservation check >2014-08-12 13:58:59,036 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) Finished HA Reservation check >2014-08-12 13:58:59,046 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-3) Init VM custom properties utilities >2014-08-12 13:58:59,047 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-3) Init device custom properties utilities >2014-08-12 13:58:59,054 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) Initializing Scheduling manager >2014-08-12 13:58:59,075 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-12 13:58:59,076 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-12 13:58:59,076 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-12 13:58:59,081 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) External scheduler disabled, discovery skipped >2014-08-12 13:58:59,082 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) Initialized Scheduling manager >2014-08-12 13:58:59,083 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-3) Initializing DWH Heart Beat >2014-08-12 13:58:59,084 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-3) DWH Heart Beat initialized >2014-08-12 13:59:01,966 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h2 >2014-08-12 13:59:02,650 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: ae552e2 >2014-08-12 13:59:02,661 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GetHardwareInfoVDSCommand, log id: ae552e2 >2014-08-12 13:59:02,672 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-12 13:59:02,680 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: UNASSIGNED not exist in string table >2014-08-12 13:59:02,681 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-12 13:59:02,682 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-12 13:59:02,684 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-12 13:59:02,685 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-12 13:59:02,686 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-12 13:59:02,686 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-12 13:59:02,687 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-12 13:59:02,688 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-12 13:59:02,688 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-12 13:59:02,690 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-12 13:59:02,690 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: DWH_STOPPED not exist in string table >2014-08-12 13:59:02,690 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: DWH_STARTED not exist in string table >2014-08-12 13:59:02,691 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: DWH_ERROR not exist in string table >2014-08-12 13:59:02,691 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-12 13:59:02,692 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-12 13:59:02,692 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-12 13:59:02,693 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-12 13:59:02,926 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-12 13:59:02,927 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is running with disabled SELinux. >2014-08-12 13:59:03,268 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: h1 >2014-08-12 13:59:03,419 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-7) [6366e0c6] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-12 13:59:09,103 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-12 13:59:09,121 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-12 13:59:09,139 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-12 13:59:09,236 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] Correlation ID: 2aa3e99e, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-12 13:59:09,341 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-12 13:59:09,352 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-12 13:59:09,377 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-12 13:59:09,394 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [2aa3e99e] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 7b101bf6 >2014-08-12 13:59:09,526 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 48 GB of free space >2014-08-12 13:59:09,641 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 48 GB of free space >2014-08-12 13:59:09,642 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [2aa3e99e] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-12 13:59:09,648 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [2aa3e99e] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 38a9a82b >2014-08-12 13:59:09,655 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [2aa3e99e] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 38a9a82b >2014-08-12 13:59:09,656 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [2aa3e99e] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 7b101bf6 >2014-08-12 13:59:09,656 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [2aa3e99e] Discovered no tasks on Storage Pool Default >2014-08-12 13:59:18,408 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-4) Running command: LoginAdminUserCommand internal: false. >2014-08-12 13:59:18,430 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-12 13:59:22,674 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-4) Running command: LoginUserCommand internal: false. >2014-08-12 14:09:24,080 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [31523a0b] Lock Acquired to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-12 14:14:51,826 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-42) Command SpmStatusVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException >2014-08-12 14:14:55,380 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-42) [46abe3f3] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-12 14:14:55,448 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-42) [46abe3f3] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-12 14:14:55,470 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-42) [46abe3f3] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-12 14:14:55,527 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-42) [46abe3f3] Correlation ID: 46abe3f3, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host h1, Error: Network error during communication with the Host.). >2014-08-12 14:16:44,759 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-33) [31523a0b] Running command: HibernateVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-12 14:16:44,825 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-33) [31523a0b] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@b776066c), log id: 3a9d536c >2014-08-12 14:16:44,828 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-33) [31523a0b] FINISH, UpdateVmDynamicDataVDSCommand, log id: 3a9d536c >2014-08-12 14:16:44,844 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-33) [31523a0b] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = d8bc9f17-167d-4311-9d78-d65c92bb8506, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = 81c9c42a-cf09-466f-abd6-c852a216dc7f, newImageDescription = ), log id: 73217d5e >2014-08-12 14:16:44,851 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-33) [31523a0b] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-12 14:16:45,275 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-33) [31523a0b] FINISH, CreateImageVDSCommand, return: 81c9c42a-cf09-466f-abd6-c852a216dc7f, log id: 73217d5e >2014-08-12 14:16:45,401 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-33) [31523a0b] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 9b21b74a-cb35-4026-88b9-4083de079920 >2014-08-12 14:16:45,404 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-33) [31523a0b] CommandMultiAsyncTasks::AttachTask: Attaching task 89b6c170-c49c-4df5-84df-a3c12a01e66f to command 9b21b74a-cb35-4026-88b9-4083de079920. >2014-08-12 14:16:45,423 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-33) [31523a0b] Adding task 89b6c170-c49c-4df5-84df-a3c12a01e66f (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-12 14:16:45,447 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-33) [31523a0b] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = ee881bce-a12e-484e-afc4-50a7dfbf6a17, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = c098dbe7-f0c0-4cc8-a658-3a05bf0f4561, newImageDescription = ), log id: 2c426d34 >2014-08-12 14:16:45,450 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-33) [31523a0b] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-12 14:16:45,975 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-33) [31523a0b] FINISH, CreateImageVDSCommand, return: c098dbe7-f0c0-4cc8-a658-3a05bf0f4561, log id: 2c426d34 >2014-08-12 14:16:46,012 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-33) [31523a0b] CommandMultiAsyncTasks::AttachTask: Attaching task 63aafd63-f3dc-4fcf-935a-d4e9bbbef0bb to command 9b21b74a-cb35-4026-88b9-4083de079920. >2014-08-12 14:16:46,026 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-33) [31523a0b] Adding task 63aafd63-f3dc-4fcf-935a-d4e9bbbef0bb (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-12 14:16:46,048 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-33) [31523a0b] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@b8a38cfa), log id: 5875db09 >2014-08-12 14:16:46,060 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-33) [31523a0b] FINISH, UpdateVmDynamicDataVDSCommand, log id: 5875db09 >2014-08-12 14:16:46,151 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-33) [31523a0b] Correlation ID: 31523a0b, Job ID: cc05869d-5d8e-420a-8904-4013f76c934e, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm was initiated by User admin (Host: h1). >2014-08-12 14:16:46,152 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-33) [31523a0b] BaseAsyncTask::startPollingTask: Starting to poll task 89b6c170-c49c-4df5-84df-a3c12a01e66f. >2014-08-12 14:16:46,152 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-33) [31523a0b] BaseAsyncTask::startPollingTask: Starting to poll task 63aafd63-f3dc-4fcf-935a-d4e9bbbef0bb. >2014-08-12 14:16:46,153 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-33) [31523a0b] Lock freed to object EngineLock [exclusiveLocks= key: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 value: VM >, sharedLocks= ] >2014-08-12 14:16:50,738 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) VM vm 14c0fa91-1814-438b-8d7a-4e7de1e1da25 moved from PreparingForHibernate --> Up >2014-08-12 14:16:54,500 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-46) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-12 14:16:54,534 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-46) SPMAsyncTask::PollTask: Polling task 89b6c170-c49c-4df5-84df-a3c12a01e66f (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-12 14:16:54,574 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-46) BaseAsyncTask::onTaskEndSuccess: Task 89b6c170-c49c-4df5-84df-a3c12a01e66f (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-12 14:16:54,575 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-46) Task with DB Task ID a19dd47b-d092-4833-80ff-864f0f1617ad and VDSM Task ID 63aafd63-f3dc-4fcf-935a-d4e9bbbef0bb is in state Polling. End action for command 9b21b74a-cb35-4026-88b9-4083de079920 will proceed when all the entitys tasks are completed. >2014-08-12 14:16:54,576 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-46) SPMAsyncTask::PollTask: Polling task 63aafd63-f3dc-4fcf-935a-d4e9bbbef0bb (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-12 14:16:54,737 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-46) BaseAsyncTask::onTaskEndSuccess: Task 63aafd63-f3dc-4fcf-935a-d4e9bbbef0bb (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-12 14:16:54,737 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-46) CommandAsyncTask::endActionIfNecessary: All tasks of command 9b21b74a-cb35-4026-88b9-4083de079920 has ended -> executing endAction >2014-08-12 14:16:54,738 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-46) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 9b21b74a-cb35-4026-88b9-4083de079920): calling endAction . >2014-08-12 14:16:54,739 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-48) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-12 14:16:54,789 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-48) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-12 14:16:54,801 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,d8bc9f17-167d-4311-9d78-d65c92bb8506,81c9c42a-cf09-466f-abd6-c852a216dc7f,ee881bce-a12e-484e-afc4-50a7dfbf6a17,c098dbe7-f0c0-4cc8-a658-3a05bf0f4561), log id: 25f404b7 >2014-08-12 14:16:54,803 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,d8bc9f17-167d-4311-9d78-d65c92bb8506,81c9c42a-cf09-466f-abd6-c852a216dc7f,ee881bce-a12e-484e-afc4-50a7dfbf6a17,c098dbe7-f0c0-4cc8-a658-3a05bf0f4561), log id: 5cc1ec7 >2014-08-12 14:16:54,859 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, HibernateBrokerVDSCommand, log id: 5cc1ec7 >2014-08-12 14:16:54,872 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, HibernateVDSCommand, log id: 25f404b7 >2014-08-12 14:16:54,874 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-12 14:16:54,874 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-12 14:16:54,889 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-48) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 63aafd63-f3dc-4fcf-935a-d4e9bbbef0bb >2014-08-12 14:16:54,890 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 63aafd63-f3dc-4fcf-935a-d4e9bbbef0bb), log id: 7c6ab552 >2014-08-12 14:16:54,892 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=63aafd63-f3dc-4fcf-935a-d4e9bbbef0bb), log id: 4547c28f >2014-08-12 14:16:54,999 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, HSMClearTaskVDSCommand, log id: 4547c28f >2014-08-12 14:16:54,999 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, SPMClearTaskVDSCommand, log id: 7c6ab552 >2014-08-12 14:16:55,013 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-48) BaseAsyncTask::removeTaskFromDB: Removed task 63aafd63-f3dc-4fcf-935a-d4e9bbbef0bb from DataBase >2014-08-12 14:16:55,013 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-48) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 89b6c170-c49c-4df5-84df-a3c12a01e66f >2014-08-12 14:16:55,014 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 89b6c170-c49c-4df5-84df-a3c12a01e66f), log id: 35fb9ce7 >2014-08-12 14:16:55,015 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=89b6c170-c49c-4df5-84df-a3c12a01e66f), log id: 19fa1d61 >2014-08-12 14:16:55,101 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, HSMClearTaskVDSCommand, log id: 19fa1d61 >2014-08-12 14:16:55,101 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-48) FINISH, SPMClearTaskVDSCommand, log id: 35fb9ce7 >2014-08-12 14:16:55,112 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-48) BaseAsyncTask::removeTaskFromDB: Removed task 89b6c170-c49c-4df5-84df-a3c12a01e66f from DataBase >2014-08-12 14:16:55,113 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-48) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 9b21b74a-cb35-4026-88b9-4083de079920 >2014-08-12 14:16:57,000 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-60) START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=14c0fa91-1814-438b-8d7a-4e7de1e1da25, force=false, secondsToWait=0, gracefully=false, reason=), log id: 43e651d3 >2014-08-12 14:16:57,011 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-60) FINISH, DestroyVDSCommand, log id: 43e651d3 >2014-08-12 14:16:57,050 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-60) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm on Host h1 is suspended. >2014-08-12 14:16:57,052 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) VM vm (14c0fa91-1814-438b-8d7a-4e7de1e1da25) is running in db and not running in VDS h1 >2014-08-12 14:16:57,053 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-60) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[14c0fa91-1814-438b-8d7a-4e7de1e1da25]), log id: 747856dc >2014-08-12 14:16:57,059 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-60) FINISH, FullListVdsCommand, return: [], log id: 747856dc >2014-08-12 14:16:57,108 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-4) [6f3a3bcd] Running command: ProcessDownVmCommand internal: true. >2014-08-12 14:17:56,352 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-7) [6366e0c6] Setting new tasks map. The map contains now 0 tasks >2014-08-12 14:17:56,353 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-7) [6366e0c6] Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-12 14:30:53,377 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-4) [7a61c02] Lock Acquired to object EngineLock [exclusiveLocks= key: de580b19-e975-426f-80de-6c7daf1f24ab value: VM >, sharedLocks= ] >2014-08-12 14:30:53,485 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [7a61c02] START, IsVmDuringInitiatingVDSCommand( vmId = de580b19-e975-426f-80de-6c7daf1f24ab), log id: 43f272d2 >2014-08-12 14:30:53,486 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [7a61c02] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 43f272d2 >2014-08-12 14:30:53,570 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-26) [7a61c02] Running command: RunVmCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-12 14:30:53,627 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-26) [7a61c02] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@d88a7fac), log id: 14f3411d >2014-08-12 14:30:53,636 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-26) [7a61c02] FINISH, UpdateVmDynamicDataVDSCommand, log id: 14f3411d >2014-08-12 14:30:53,651 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-26) [7a61c02] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, vm=VM [vm4]), log id: 709538ec >2014-08-12 14:30:53,662 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-26) [7a61c02] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, vm=VM [vm4]), log id: 7153fcb5 >2014-08-12 14:30:53,779 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-26) [7a61c02] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=de580b19-e975-426f-80de-6c7daf1f24ab,acpiEnable=true,custom={device_90697f10-7597-4bd5-bb38-b5c2e2c9b355device_68d375f1-7d6b-4a64-a0e0-3445dcf576f7device_d807c9e6-0c2e-469a-abe2-e5e3d47b4269=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=d807c9e6-0c2e-469a-abe2-e5e3d47b4269, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_90697f10-7597-4bd5-bb38-b5c2e2c9b355device_68d375f1-7d6b-4a64-a0e0-3445dcf576f7device_d807c9e6-0c2e-469a-abe2-e5e3d47b4269device_3ba6c745-d84d-40b3-a2e8-021b894fcd2edevice_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_90697f10-7597-4bd5-bb38-b5c2e2c9b355=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=90697f10-7597-4bd5-bb38-b5c2e2c9b355, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_90697f10-7597-4bd5-bb38-b5c2e2c9b355device_68d375f1-7d6b-4a64-a0e0-3445dcf576f7device_d807c9e6-0c2e-469a-abe2-e5e3d47b4269device_3ba6c745-d84d-40b3-a2e8-021b894fcd2e=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=3ba6c745-d84d-40b3-a2e8-021b894fcd2e, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_90697f10-7597-4bd5-bb38-b5c2e2c9b355device_68d375f1-7d6b-4a64-a0e0-3445dcf576f7=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=68d375f1-7d6b-4a64-a0e0-3445dcf576f7, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={ram=65536, vram=32768, heads=1}, device=qxl, type=video, deviceId=b3b178bf-b83c-4659-a950-b4e8762616e5}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=5e9cb4a3-8653-4e26-80c9-ff9a205bacea}, {shared=false, index=0, volumeID=3938bfe7-a921-4b4e-b337-fed4aa931c88, propagateErrors=off, format=raw, type=disk, iface=virtio, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageID=016a7ddb-9007-4925-ae72-202649686c94, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=016a7ddb-9007-4925-ae72-202649686c94}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:54, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=85f4913e-164d-4963-a4f3-c9761543de9e}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=271440b0-d17a-4d84-be46-df27b769e570}, {address={bus=0x00, domain=0x0000, slot=0x05, type=pci, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2}],vmName=vm4,cpuType=SandyBridge,fileTransferEnable=true >2014-08-12 14:30:53,814 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-26) [7a61c02] FINISH, CreateVDSCommand, log id: 7153fcb5 >2014-08-12 14:30:53,843 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-26) [7a61c02] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 709538ec >2014-08-12 14:30:53,844 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-26) [7a61c02] Lock freed to object EngineLock [exclusiveLocks= key: de580b19-e975-426f-80de-6c7daf1f24ab value: VM >, sharedLocks= ] >2014-08-12 14:30:53,860 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-26) [7a61c02] Correlation ID: 7a61c02, Job ID: f12a90c4-460b-406f-8c31-76a43b4f4dcd, Call Stack: null, Custom Event ID: -1, Message: VM vm4 was started by admin (Host: h1). >2014-08-12 14:30:55,356 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) VM vm4 de580b19-e975-426f-80de-6c7daf1f24ab moved from WaitForLaunch --> PoweringUp >2014-08-12 14:30:55,357 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-93) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[de580b19-e975-426f-80de-6c7daf1f24ab]), log id: 1f62ee66 >2014-08-12 14:30:55,367 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-93) FINISH, FullListVdsCommand, return: [{displaySecurePort=5900, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.35.0.105, pauseCode=NOERR, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, acpiEnable=true, custom={device_90697f10-7597-4bd5-bb38-b5c2e2c9b355device_68d375f1-7d6b-4a64-a0e0-3445dcf576f7device_d807c9e6-0c2e-469a-abe2-e5e3d47b4269=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=d807c9e6-0c2e-469a-abe2-e5e3d47b4269, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_90697f10-7597-4bd5-bb38-b5c2e2c9b355device_68d375f1-7d6b-4a64-a0e0-3445dcf576f7device_d807c9e6-0c2e-469a-abe2-e5e3d47b4269device_3ba6c745-d84d-40b3-a2e8-021b894fcd2edevice_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_90697f10-7597-4bd5-bb38-b5c2e2c9b355=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=90697f10-7597-4bd5-bb38-b5c2e2c9b355, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_90697f10-7597-4bd5-bb38-b5c2e2c9b355device_68d375f1-7d6b-4a64-a0e0-3445dcf576f7device_d807c9e6-0c2e-469a-abe2-e5e3d47b4269device_3ba6c745-d84d-40b3-a2e8-021b894fcd2e=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=3ba6c745-d84d-40b3-a2e8-021b894fcd2e, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_90697f10-7597-4bd5-bb38-b5c2e2c9b355device_68d375f1-7d6b-4a64-a0e0-3445dcf576f7=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=68d375f1-7d6b-4a64-a0e0-3445dcf576f7, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=11120, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@238a6fad, vmName=vm4, fileTransferEnable=true, cpuType=SandyBridge}], log id: 1f62ee66 >2014-08-12 14:30:55,379 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Received a memballoon Device without an address when processing VM de580b19-e975-426f-80de-6c7daf1f24ab devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-08-12 14:30:55,386 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Received a spice Device without an address when processing VM de580b19-e975-426f-80de-6c7daf1f24ab devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5900, type=graphics} >2014-08-12 14:31:54,334 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) VM vm4 de580b19-e975-426f-80de-6c7daf1f24ab moved from PoweringUp --> Up >2014-08-12 14:31:54,497 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-54) Correlation ID: 7a61c02, Job ID: f12a90c4-460b-406f-8c31-76a43b4f4dcd, Call Stack: null, Custom Event ID: -1, Message: VM vm4 started on Host h1 >2014-08-12 14:33:41,816 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (http--0.0.0.0-8080-4) [3a8ea7ec] Running command: SetVmTicketCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group CONNECT_TO_VM with role type USER >2014-08-12 14:33:41,820 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-4) [3a8ea7ec] START, SetVmTicketVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, ticket=uvBRXaxogcf6, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 3a140a51 >2014-08-12 14:33:41,850 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-4) [3a8ea7ec] FINISH, SetVmTicketVDSCommand, log id: 3a140a51 >2014-08-12 14:33:41,886 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [3a8ea7ec] Correlation ID: 3a8ea7ec, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM vm4 >2014-08-12 14:34:20,148 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (http--0.0.0.0-8080-4) [788dd3a4] Running command: SetVmTicketCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group CONNECT_TO_VM with role type USER >2014-08-12 14:34:20,152 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-4) [788dd3a4] START, SetVmTicketVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, ticket=0Rm/WXQmh98n, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 63d07da9 >2014-08-12 14:34:20,164 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-4) [788dd3a4] FINISH, SetVmTicketVDSCommand, log id: 63d07da9 >2014-08-12 14:34:20,213 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [788dd3a4] Correlation ID: 788dd3a4, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM vm4 >2014-08-12 14:42:00,144 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (http--0.0.0.0-8080-4) [1a0afa58] Running command: SetVmTicketCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group CONNECT_TO_VM with role type USER >2014-08-12 14:42:00,146 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-4) [1a0afa58] START, SetVmTicketVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, ticket=XEOnODOURYWM, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: a5f24d1 >2014-08-12 14:42:00,158 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-4) [1a0afa58] FINISH, SetVmTicketVDSCommand, log id: a5f24d1 >2014-08-12 14:42:00,184 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [1a0afa58] Correlation ID: 1a0afa58, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM vm4 >2014-08-12 14:42:51,291 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (http--0.0.0.0-8080-4) [304ad1b1] Lock Acquired to object EngineLock [exclusiveLocks= key: vm3 value: VM_NAME >, sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-12 14:42:51,404 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (http--0.0.0.0-8080-4) [304ad1b1] Running command: UpdateVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group EDIT_VM_PROPERTIES with role type USER >2014-08-12 14:42:51,446 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [304ad1b1] Correlation ID: 304ad1b1, Job ID: 3f3273cb-e949-427c-b73c-ca63b8dda083, Call Stack: null, Custom Event ID: -1, Message: VM vm3 configuration was updated by admin. >2014-08-12 14:42:51,454 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (http--0.0.0.0-8080-4) [304ad1b1] Lock freed to object EngineLock [exclusiveLocks= key: vm3 value: VM_NAME >, sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-12 14:42:55,475 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-4) [38d2b0f3] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-12 14:42:55,504 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [38d2b0f3] START, IsVmDuringInitiatingVDSCommand( vmId = 5b1ee280-4251-489a-899c-8e9dac73c327), log id: 574b3c88 >2014-08-12 14:42:55,505 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [38d2b0f3] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 574b3c88 >2014-08-12 14:42:55,613 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-14) [38d2b0f3] Running command: RunVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-12 14:42:55,682 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-14) [38d2b0f3] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = ISCSI, connectionList = [{ id: bd327084-78e4-4b84-9a74-9cea2d43c971, connection: 10.35.16.25, iqn: iqn.2011-05.com.redhat:achub-iscsi, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 4239751c >2014-08-12 14:42:56,091 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-14) [38d2b0f3] FINISH, ConnectStorageServerVDSCommand, return: {bd327084-78e4-4b84-9a74-9cea2d43c971=0}, log id: 4239751c >2014-08-12 14:42:56,096 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-14) [38d2b0f3] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@eae087ea), log id: 2cba967d >2014-08-12 14:42:56,242 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-14) [38d2b0f3] FINISH, UpdateVmDynamicDataVDSCommand, log id: 2cba967d >2014-08-12 14:42:56,245 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-14) [38d2b0f3] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, vm=VM [vm3]), log id: 4676680a >2014-08-12 14:42:56,249 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-14) [38d2b0f3] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, vm=VM [vm3]), log id: 19f57a1e >2014-08-12 14:42:56,271 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-14) [38d2b0f3] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,timeOffset=0,transparentHugePages=true,vmId=5b1ee280-4251-489a-899c-8e9dac73c327,acpiEnable=true,custom={device_baa98cda-dcea-4609-b1f9-907527c03df5device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_9bfd764d-3d78-4523-80ba-176be32c6f6edevice_f7f57a89-ed4d-42fb-9aa2-b2a8606df16fdevice_44123eda-783f-40d1-a5e4-e3cccf33a24e=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=44123eda-783f-40d1-a5e4-e3cccf33a24e, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_baa98cda-dcea-4609-b1f9-907527c03df5=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=baa98cda-dcea-4609-b1f9-907527c03df5, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_baa98cda-dcea-4609-b1f9-907527c03df5device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_9bfd764d-3d78-4523-80ba-176be32c6f6e=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=9bfd764d-3d78-4523-80ba-176be32c6f6e, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_baa98cda-dcea-4609-b1f9-907527c03df5device_96c8ccf7-eae6-4354-ac32-442f636eee8c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_baa98cda-dcea-4609-b1f9-907527c03df5device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_9bfd764d-3d78-4523-80ba-176be32c6f6edevice_f7f57a89-ed4d-42fb-9aa2-b2a8606df16f=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=f7f57a89-ed4d-42fb-9aa2-b2a8606df16f, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=vnc,smartcardEnable=false,bootMenuEnable=false,tabletEnable=true,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{specParams={vram=32768, heads=1}, device=cirrus, type=video, deviceId=2473b074-1cf7-46bc-b515-4e4036653830}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=f76be01f-4e52-4d67-8e1b-0856337e9929}, {shared=false, index=0, volumeID=26308abd-8540-410e-8018-58cd2157084d, propagateErrors=off, format=cow, type=disk, iface=virtio, bootOrder=1, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, imageID=b5576e45-7f13-465f-b26f-05d52faf28d1, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=b5576e45-7f13-465f-b26f-05d52faf28d1}, {shared=false, iface=virtio, GUID=3600144f09dbd050000004e1994c60005, address={bus=0x00, domain=0x0000, slot=0x07, type=pci, function=0x0}, specParams={}, optional=false, propagateErrors=off, device=lun, format=raw, type=disk, readonly=false, deviceId=bdca3776-6597-4323-84f1-4ea75f7d269d}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:53, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=68e93502-4ad4-42a0-9a23-dfa3f67090f5}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=403887de-d0b8-4108-8b43-b4b5d5413859}, {address={bus=0x00, domain=0x0000, slot=0x05, type=pci, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c}],vmName=vm3,cpuType=SandyBridge >2014-08-12 14:42:56,304 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-14) [38d2b0f3] FINISH, CreateVDSCommand, log id: 19f57a1e >2014-08-12 14:42:56,333 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-14) [38d2b0f3] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 4676680a >2014-08-12 14:42:56,334 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-14) [38d2b0f3] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-12 14:42:56,341 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-14) [38d2b0f3] Correlation ID: 38d2b0f3, Job ID: 85f58368-3721-4cc1-a778-a31ab2da718d, Call Stack: null, Custom Event ID: -1, Message: VM vm3 was started by admin (Host: h1). >2014-08-12 14:42:59,049 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from WaitForLaunch --> PoweringUp >2014-08-12 14:42:59,050 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-1) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: 397c48aa >2014-08-12 14:42:59,064 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-1) FINISH, FullListVdsCommand, return: [{displaySecurePort=-1, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.35.0.105, pauseCode=NOERR, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, timeOffset=0, transparentHugePages=true, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, acpiEnable=true, custom={device_baa98cda-dcea-4609-b1f9-907527c03df5device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_9bfd764d-3d78-4523-80ba-176be32c6f6edevice_f7f57a89-ed4d-42fb-9aa2-b2a8606df16fdevice_44123eda-783f-40d1-a5e4-e3cccf33a24e=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=44123eda-783f-40d1-a5e4-e3cccf33a24e, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_baa98cda-dcea-4609-b1f9-907527c03df5=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=baa98cda-dcea-4609-b1f9-907527c03df5, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_baa98cda-dcea-4609-b1f9-907527c03df5device_96c8ccf7-eae6-4354-ac32-442f636eee8c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_baa98cda-dcea-4609-b1f9-907527c03df5device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_9bfd764d-3d78-4523-80ba-176be32c6f6e=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=9bfd764d-3d78-4523-80ba-176be32c6f6e, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_baa98cda-dcea-4609-b1f9-907527c03df5device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_9bfd764d-3d78-4523-80ba-176be32c6f6edevice_f7f57a89-ed4d-42fb-9aa2-b2a8606df16f=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=f7f57a89-ed4d-42fb-9aa2-b2a8606df16f, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=5901, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=vnc, pid=12073, smartcardEnable=false, bootMenuEnable=false, tabletEnable=true, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@41f7d83, vmName=vm3, cpuType=SandyBridge}], log id: 397c48aa >2014-08-12 14:42:59,072 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Received a memballoon Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-08-12 14:42:59,073 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Received a vnc Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {port=5901, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, displayIp=10.35.0.105}, device=vnc, type=graphics} >2014-08-12 14:43:07,148 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (http--0.0.0.0-8080-4) [16a3018e] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group CONNECT_TO_VM with role type USER >2014-08-12 14:43:07,151 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-4) [16a3018e] START, SetVmTicketVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, ticket=mVf7/wUM5g3X, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 6f2ffbb1 >2014-08-12 14:43:07,163 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-4) [16a3018e] FINISH, SetVmTicketVDSCommand, log id: 6f2ffbb1 >2014-08-12 14:43:07,190 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [16a3018e] Correlation ID: 16a3018e, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM vm3 >2014-08-12 14:43:17,996 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-56) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin is connected to VM vm3. >2014-08-12 14:43:58,627 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from PoweringUp --> Up >2014-08-12 14:43:58,702 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) Correlation ID: 38d2b0f3, Job ID: 85f58368-3721-4cc1-a778-a31ab2da718d, Call Stack: null, Custom Event ID: -1, Message: VM vm3 started on Host h1 >2014-08-12 14:44:38,326 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin got disconnected from VM vm3. >2014-08-12 14:44:40,437 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-38) [28200fb4] Running command: StopVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-12 14:44:40,442 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-38) [28200fb4] START, DestroyVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, force=false, secondsToWait=0, gracefully=false, reason=), log id: 74f1cc9d >2014-08-12 14:44:40,452 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-38) [28200fb4] START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, force=false, secondsToWait=0, gracefully=false, reason=), log id: 61c7f4e7 >2014-08-12 14:44:40,713 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-38) [28200fb4] FINISH, DestroyVDSCommand, log id: 61c7f4e7 >2014-08-12 14:44:40,727 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-38) [28200fb4] FINISH, DestroyVmVDSCommand, return: Down, log id: 74f1cc9d >2014-08-12 14:44:40,743 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-38) [28200fb4] Correlation ID: 28200fb4, Job ID: 0ddc92d8-cc87-4856-ba00-68f7a0f8cf45, Call Stack: null, Custom Event ID: -1, Message: VM vm3 powered off by admin (Host: h1) (Reason: Not Specified). >2014-08-12 14:44:40,761 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-40) [1bd328f0] Running command: ProcessDownVmCommand internal: true. >2014-08-12 14:58:58,787 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-82) Principal admin::internal synchronized >2014-08-12 14:58:59,033 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-91) Attempting to update VMs/Templates Ovf. >2014-08-12 14:58:59,373 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-91) [4a32b476] Lock Acquired to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-12 14:58:59,377 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-91) [4a32b476] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-12 14:58:59,412 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [4a32b476] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 5353e718 >2014-08-12 14:58:59,413 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [4a32b476] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-12 14:58:59,414 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [4a32b476] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Mon Aug 11 19:28:43 IDT 2014"} > >2014-08-12 14:58:59,522 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [4a32b476] FINISH, SetVolumeDescriptionVDSCommand, log id: 5353e718 >2014-08-12 14:58:59,597 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-12 14:58:59,602 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 285d50 >2014-08-12 14:58:59,602 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] -- executeVdsBrokerCommand: >2014-08-12 14:58:59,603 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > size=71,680 > >2014-08-12 14:59:01,321 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] FINISH, UploadStreamVDSCommand, log id: 285d50 >2014-08-12 14:59:01,332 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-91) [5ac59299] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 0185e65f-2bcf-4319-9032-a9002a91e6dc >2014-08-12 14:59:01,333 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-91) [5ac59299] CommandMultiAsyncTasks::AttachTask: Attaching task fbb12ea9-1c96-4794-8c38-41caecbff3e6 to command 0185e65f-2bcf-4319-9032-a9002a91e6dc. >2014-08-12 14:59:01,394 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-91) [5ac59299] Adding task fbb12ea9-1c96-4794-8c38-41caecbff3e6 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-12 14:59:01,411 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-12 14:59:01,416 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 50b9651b >2014-08-12 14:59:01,417 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-12 14:59:01,418 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Tue Aug 12 14:58:59 IDT 2014","Size":71680} > >2014-08-12 14:59:01,548 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] FINISH, SetVolumeDescriptionVDSCommand, log id: 50b9651b >2014-08-12 14:59:01,562 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 2e450a13 >2014-08-12 14:59:01,563 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-12 14:59:01,564 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Mon Aug 11 19:28:43 IDT 2014"} > >2014-08-12 14:59:01,647 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [5ac59299] FINISH, SetVolumeDescriptionVDSCommand, log id: 2e450a13 >2014-08-12 14:59:01,709 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-12 14:59:01,711 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 96eb673 >2014-08-12 14:59:01,712 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] -- executeVdsBrokerCommand: >2014-08-12 14:59:01,713 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > size=71,680 > >2014-08-12 14:59:03,376 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] FINISH, UploadStreamVDSCommand, log id: 96eb673 >2014-08-12 14:59:03,415 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] CommandMultiAsyncTasks::AttachTask: Attaching task fa3b8e78-5501-480c-8a7e-7707b3b90526 to command 0185e65f-2bcf-4319-9032-a9002a91e6dc. >2014-08-12 14:59:03,432 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] Adding task fa3b8e78-5501-480c-8a7e-7707b3b90526 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-12 14:59:03,448 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-12 14:59:03,449 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-12 14:59:03,449 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-12 14:59:03,450 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-12 14:59:03,450 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-12 14:59:03,452 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 4d95ccfa >2014-08-12 14:59:03,454 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-12 14:59:03,454 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Tue Aug 12 14:58:59 IDT 2014","Size":71680} > >2014-08-12 14:59:03,561 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] FINISH, SetVolumeDescriptionVDSCommand, log id: 4d95ccfa >2014-08-12 14:59:03,572 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] BaseAsyncTask::startPollingTask: Starting to poll task fbb12ea9-1c96-4794-8c38-41caecbff3e6. >2014-08-12 14:59:03,573 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] BaseAsyncTask::startPollingTask: Starting to poll task fa3b8e78-5501-480c-8a7e-7707b3b90526. >2014-08-12 14:59:04,810 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-42) [46abe3f3] Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-12 14:59:04,822 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-42) [46abe3f3] SPMAsyncTask::PollTask: Polling task fa3b8e78-5501-480c-8a7e-7707b3b90526 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-12 14:59:04,823 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-42) [46abe3f3] BaseAsyncTask::onTaskEndSuccess: Task fa3b8e78-5501-480c-8a7e-7707b3b90526 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-12 14:59:04,824 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-42) [46abe3f3] Task with DB Task ID 6433788a-2a16-4637-acf9-1bff5e1074a3 and VDSM Task ID fbb12ea9-1c96-4794-8c38-41caecbff3e6 is in state Polling. End action for command 0185e65f-2bcf-4319-9032-a9002a91e6dc will proceed when all the entitys tasks are completed. >2014-08-12 14:59:04,825 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-42) [46abe3f3] SPMAsyncTask::PollTask: Polling task fbb12ea9-1c96-4794-8c38-41caecbff3e6 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-12 14:59:04,826 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-42) [46abe3f3] BaseAsyncTask::onTaskEndSuccess: Task fbb12ea9-1c96-4794-8c38-41caecbff3e6 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-12 14:59:04,827 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-42) [46abe3f3] CommandAsyncTask::endActionIfNecessary: All tasks of command 0185e65f-2bcf-4319-9032-a9002a91e6dc has ended -> executing endAction >2014-08-12 14:59:04,828 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-42) [46abe3f3] CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 0185e65f-2bcf-4319-9032-a9002a91e6dc): calling endAction . >2014-08-12 14:59:04,828 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) [46abe3f3] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-08-12 14:59:04,847 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-50) [4a32b476] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-08-12 14:59:04,848 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-50) [4a32b476] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-12 14:59:04,849 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-50) [4a32b476] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-12 14:59:04,849 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-50) [4a32b476] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-12 14:59:04,850 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-50) [4a32b476] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-12 14:59:04,850 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-50) [4a32b476] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-12 14:59:04,851 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) [4a32b476] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-08-12 14:59:04,852 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) [4a32b476] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-08-12 14:59:04,852 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) [4a32b476] SPMAsyncTask::ClearAsyncTask: Attempting to clear task fbb12ea9-1c96-4794-8c38-41caecbff3e6 >2014-08-12 14:59:04,853 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [4a32b476] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = fbb12ea9-1c96-4794-8c38-41caecbff3e6), log id: 4427f9c7 >2014-08-12 14:59:04,854 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [4a32b476] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=fbb12ea9-1c96-4794-8c38-41caecbff3e6), log id: 1c7f9ac6 >2014-08-12 14:59:04,917 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [4a32b476] FINISH, HSMClearTaskVDSCommand, log id: 1c7f9ac6 >2014-08-12 14:59:04,917 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [4a32b476] FINISH, SPMClearTaskVDSCommand, log id: 4427f9c7 >2014-08-12 14:59:04,955 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) [4a32b476] BaseAsyncTask::removeTaskFromDB: Removed task fbb12ea9-1c96-4794-8c38-41caecbff3e6 from DataBase >2014-08-12 14:59:04,956 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) [4a32b476] SPMAsyncTask::ClearAsyncTask: Attempting to clear task fa3b8e78-5501-480c-8a7e-7707b3b90526 >2014-08-12 14:59:04,956 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [4a32b476] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = fa3b8e78-5501-480c-8a7e-7707b3b90526), log id: 5c46f54b >2014-08-12 14:59:04,959 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [4a32b476] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=fa3b8e78-5501-480c-8a7e-7707b3b90526), log id: 36923e41 >2014-08-12 14:59:05,035 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [4a32b476] FINISH, HSMClearTaskVDSCommand, log id: 36923e41 >2014-08-12 14:59:05,036 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [4a32b476] FINISH, SPMClearTaskVDSCommand, log id: 5c46f54b >2014-08-12 14:59:05,046 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) [4a32b476] BaseAsyncTask::removeTaskFromDB: Removed task fa3b8e78-5501-480c-8a7e-7707b3b90526 from DataBase >2014-08-12 14:59:05,047 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) [4a32b476] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 0185e65f-2bcf-4319-9032-a9002a91e6dc >2014-08-12 15:00:26,391 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-30) Setting new tasks map. The map contains now 0 tasks >2014-08-12 15:00:26,392 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-30) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-12 15:08:24,204 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [549910b5] Lock Acquired to object EngineLock [exclusiveLocks= key: de580b19-e975-426f-80de-6c7daf1f24ab value: VM >, sharedLocks= ] >2014-08-12 15:08:24,266 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-46) Command bf7b2dfc-0561-4ff6-bfd8-cd02687a53b8 persisting async task placeholder for child command 5c885217-73e8-4afd-8e86-0cbb5cdd8605 >2014-08-12 15:08:24,373 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-46) Running command: CreateAllSnapshotsFromVmCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER >2014-08-12 15:08:24,432 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-46) [6e7285ae] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-08-12 15:08:24,438 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-46) [6e7285ae] START, CreateSnapshotVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 016a7ddb-9007-4925-ae72-202649686c94, imageSizeInBytes = 53687091200, volumeFormat = COW, newImageId = c70c663c-765a-4808-b98d-af453f8cc170, newImageDescription = , imageId = 3938bfe7-a921-4b4e-b337-fed4aa931c88, sourceImageGroupId = 016a7ddb-9007-4925-ae72-202649686c94), log id: 25994626 >2014-08-12 15:08:24,439 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-46) [6e7285ae] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-12 15:08:24,857 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-46) [6e7285ae] FINISH, CreateSnapshotVDSCommand, return: c70c663c-765a-4808-b98d-af453f8cc170, log id: 25994626 >2014-08-12 15:08:25,032 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-46) [6e7285ae] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command bf7b2dfc-0561-4ff6-bfd8-cd02687a53b8 >2014-08-12 15:08:25,033 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-46) [6e7285ae] CommandMultiAsyncTasks::AttachTask: Attaching task 6c5871d9-4196-492f-9125-9aac80b4194d to command bf7b2dfc-0561-4ff6-bfd8-cd02687a53b8. >2014-08-12 15:08:25,053 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) [6e7285ae] Adding task 6c5871d9-4196-492f-9125-9aac80b4194d (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-12 15:08:25,117 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-46) START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 34feecc4-b988-4733-94f6-5debb661f0b6, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = 852405df-1744-4540-8f61-ae623424bd4f, newImageDescription = ), log id: 6761c029 >2014-08-12 15:08:25,118 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-46) -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-12 15:08:25,561 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-46) FINISH, CreateImageVDSCommand, return: 852405df-1744-4540-8f61-ae623424bd4f, log id: 6761c029 >2014-08-12 15:08:25,638 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-46) CommandMultiAsyncTasks::AttachTask: Attaching task 2628a77a-a657-460e-ab14-d3b001a25586 to command bf7b2dfc-0561-4ff6-bfd8-cd02687a53b8. >2014-08-12 15:08:25,658 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) Adding task 2628a77a-a657-460e-ab14-d3b001a25586 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-12 15:08:25,667 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-46) START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = f6edaf20-e507-471c-b6c1-487c35e1d7cf, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = f41f7917-8d0d-4168-9586-fa21c97aae3e, newImageDescription = ), log id: 63bcf2c1 >2014-08-12 15:08:25,668 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-46) -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-12 15:08:26,420 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-46) FINISH, CreateImageVDSCommand, return: f41f7917-8d0d-4168-9586-fa21c97aae3e, log id: 63bcf2c1 >2014-08-12 15:08:26,514 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-46) CommandMultiAsyncTasks::AttachTask: Attaching task e3121919-954a-417e-8ec3-64414eb73cb3 to command bf7b2dfc-0561-4ff6-bfd8-cd02687a53b8. >2014-08-12 15:08:26,536 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-46) Adding task e3121919-954a-417e-8ec3-64414eb73cb3 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-12 15:08:26,586 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-46) Correlation ID: 549910b5, Job ID: 6916fd30-604d-45bc-801d-1ecb12b6b186, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'snap1' creation for VM 'vm4' was initiated by admin. >2014-08-12 15:08:26,587 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) BaseAsyncTask::startPollingTask: Starting to poll task 6c5871d9-4196-492f-9125-9aac80b4194d. >2014-08-12 15:08:26,587 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) BaseAsyncTask::startPollingTask: Starting to poll task 2628a77a-a657-460e-ab14-d3b001a25586. >2014-08-12 15:08:26,587 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-46) BaseAsyncTask::startPollingTask: Starting to poll task e3121919-954a-417e-8ec3-64414eb73cb3. >2014-08-12 15:08:34,847 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-96) Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now >2014-08-12 15:08:34,854 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-96) SPMAsyncTask::PollTask: Polling task 2628a77a-a657-460e-ab14-d3b001a25586 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-12 15:08:34,877 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-96) BaseAsyncTask::onTaskEndSuccess: Task 2628a77a-a657-460e-ab14-d3b001a25586 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-12 15:08:34,878 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-96) Task with DB Task ID d2088514-15fd-4e72-b7ed-6d10b7f27d0e and VDSM Task ID 6c5871d9-4196-492f-9125-9aac80b4194d is in state Polling. End action for command bf7b2dfc-0561-4ff6-bfd8-cd02687a53b8 will proceed when all the entitys tasks are completed. >2014-08-12 15:08:34,878 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-96) SPMAsyncTask::PollTask: Polling task e3121919-954a-417e-8ec3-64414eb73cb3 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-12 15:08:34,885 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-96) BaseAsyncTask::onTaskEndSuccess: Task e3121919-954a-417e-8ec3-64414eb73cb3 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-12 15:08:34,886 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-96) Task with DB Task ID d2088514-15fd-4e72-b7ed-6d10b7f27d0e and VDSM Task ID 6c5871d9-4196-492f-9125-9aac80b4194d is in state Polling. End action for command bf7b2dfc-0561-4ff6-bfd8-cd02687a53b8 will proceed when all the entitys tasks are completed. >2014-08-12 15:08:34,886 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-96) SPMAsyncTask::PollTask: Polling task 6c5871d9-4196-492f-9125-9aac80b4194d (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-12 15:08:34,893 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-96) BaseAsyncTask::onTaskEndSuccess: Task 6c5871d9-4196-492f-9125-9aac80b4194d (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-12 15:08:34,894 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-96) CommandAsyncTask::endActionIfNecessary: All tasks of command bf7b2dfc-0561-4ff6-bfd8-cd02687a53b8 has ended -> executing endAction >2014-08-12 15:08:34,894 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-96) CommandAsyncTask::endAction: Ending action for 3 tasks (command ID: bf7b2dfc-0561-4ff6-bfd8-cd02687a53b8): calling endAction . >2014-08-12 15:08:34,895 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-16) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction CreateAllSnapshotsFromVm, executionIndex: 0 >2014-08-12 15:08:34,943 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-16) Ending command successfully: org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand >2014-08-12 15:08:34,965 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-16) START, SnapshotVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab), log id: 41fe5ea3 >2014-08-12 15:08:37,320 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] VM vm4 de580b19-e975-426f-80de-6c7daf1f24ab moved from Up --> Paused >2014-08-12 15:08:44,254 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-16) FINISH, SnapshotVDSCommand, log id: 41fe5ea3 >2014-08-12 15:08:44,303 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-16) [6e7285ae] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotCommand >2014-08-12 15:08:44,321 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-16) [6e7285ae] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 016a7ddb-9007-4925-ae72-202649686c94, imageId = c70c663c-765a-4808-b98d-af453f8cc170), log id: 4788b4bf >2014-08-12 15:08:44,351 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-16) [6e7285ae] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@2c2a358, log id: 4788b4bf >2014-08-12 15:08:44,383 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-16) Lock freed to object EngineLock [exclusiveLocks= key: de580b19-e975-426f-80de-6c7daf1f24ab value: VM >, sharedLocks= ] >2014-08-12 15:08:44,410 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-16) Correlation ID: 549910b5, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'snap1' creation for VM 'vm4' has been completed. >2014-08-12 15:08:44,411 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-16) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type CreateAllSnapshotsFromVm completed, handling the result. >2014-08-12 15:08:44,412 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-16) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type CreateAllSnapshotsFromVm succeeded, clearing tasks. >2014-08-12 15:08:44,426 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-16) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 6c5871d9-4196-492f-9125-9aac80b4194d >2014-08-12 15:08:44,427 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-16) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 6c5871d9-4196-492f-9125-9aac80b4194d), log id: 40dbce0c >2014-08-12 15:08:44,430 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-16) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=6c5871d9-4196-492f-9125-9aac80b4194d), log id: 622bb867 >2014-08-12 15:08:44,695 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-16) FINISH, HSMClearTaskVDSCommand, log id: 622bb867 >2014-08-12 15:08:44,696 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-16) FINISH, SPMClearTaskVDSCommand, log id: 40dbce0c >2014-08-12 15:08:44,708 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-16) BaseAsyncTask::removeTaskFromDB: Removed task 6c5871d9-4196-492f-9125-9aac80b4194d from DataBase >2014-08-12 15:08:44,709 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-16) SPMAsyncTask::ClearAsyncTask: Attempting to clear task e3121919-954a-417e-8ec3-64414eb73cb3 >2014-08-12 15:08:44,710 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-16) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = e3121919-954a-417e-8ec3-64414eb73cb3), log id: 7c31c338 >2014-08-12 15:08:44,712 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-16) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=e3121919-954a-417e-8ec3-64414eb73cb3), log id: 2707d13b >2014-08-12 15:08:45,125 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-16) FINISH, HSMClearTaskVDSCommand, log id: 2707d13b >2014-08-12 15:08:45,126 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-16) FINISH, SPMClearTaskVDSCommand, log id: 7c31c338 >2014-08-12 15:08:45,238 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-16) BaseAsyncTask::removeTaskFromDB: Removed task e3121919-954a-417e-8ec3-64414eb73cb3 from DataBase >2014-08-12 15:08:45,239 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-16) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 2628a77a-a657-460e-ab14-d3b001a25586 >2014-08-12 15:08:45,240 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-16) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 2628a77a-a657-460e-ab14-d3b001a25586), log id: 52b15dfe >2014-08-12 15:08:45,243 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-16) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=2628a77a-a657-460e-ab14-d3b001a25586), log id: 7a91f91b >2014-08-12 15:08:45,486 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-16) FINISH, HSMClearTaskVDSCommand, log id: 7a91f91b >2014-08-12 15:08:45,487 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-16) FINISH, SPMClearTaskVDSCommand, log id: 52b15dfe >2014-08-12 15:08:45,511 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-16) BaseAsyncTask::removeTaskFromDB: Removed task 2628a77a-a657-460e-ab14-d3b001a25586 from DataBase >2014-08-12 15:08:45,512 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-16) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity bf7b2dfc-0561-4ff6-bfd8-cd02687a53b8 >2014-08-12 15:08:46,527 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) VM vm4 de580b19-e975-426f-80de-6c7daf1f24ab moved from Paused --> Up >2014-08-12 15:08:46,529 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-27) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[de580b19-e975-426f-80de-6c7daf1f24ab]), log id: 211edc3b >2014-08-12 15:08:46,540 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-27) FINISH, FullListVdsCommand, return: [{displaySecurePort=5900, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.35.0.105, pauseCode=NOERR, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, acpiEnable=true, custom={device_90697f10-7597-4bd5-bb38-b5c2e2c9b355device_68d375f1-7d6b-4a64-a0e0-3445dcf576f7device_d807c9e6-0c2e-469a-abe2-e5e3d47b4269=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=d807c9e6-0c2e-469a-abe2-e5e3d47b4269, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_90697f10-7597-4bd5-bb38-b5c2e2c9b355device_68d375f1-7d6b-4a64-a0e0-3445dcf576f7device_d807c9e6-0c2e-469a-abe2-e5e3d47b4269device_3ba6c745-d84d-40b3-a2e8-021b894fcd2edevice_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_90697f10-7597-4bd5-bb38-b5c2e2c9b355=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=90697f10-7597-4bd5-bb38-b5c2e2c9b355, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_90697f10-7597-4bd5-bb38-b5c2e2c9b355device_68d375f1-7d6b-4a64-a0e0-3445dcf576f7device_d807c9e6-0c2e-469a-abe2-e5e3d47b4269device_3ba6c745-d84d-40b3-a2e8-021b894fcd2e=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=3ba6c745-d84d-40b3-a2e8-021b894fcd2e, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_90697f10-7597-4bd5-bb38-b5c2e2c9b355device_68d375f1-7d6b-4a64-a0e0-3445dcf576f7=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=68d375f1-7d6b-4a64-a0e0-3445dcf576f7, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=11120, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@710cb679, vmName=vm4, fileTransferEnable=true, cpuType=SandyBridge}], log id: 211edc3b >2014-08-12 15:08:46,550 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Received a memballoon Device without an address when processing VM de580b19-e975-426f-80de-6c7daf1f24ab devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-08-12 15:08:46,551 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Received a spice Device without an address when processing VM de580b19-e975-426f-80de-6c7daf1f24ab devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5900, type=graphics} >2014-08-12 15:09:56,396 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-29) Setting new tasks map. The map contains now 0 tasks >2014-08-12 15:09:56,398 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-29) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-12 15:10:23,663 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (http--0.0.0.0-8080-4) [6ea7e4a6] Lock Acquired to object EngineLock [exclusiveLocks= key: de580b19-e975-426f-80de-6c7daf1f24ab value: VM >key: vm_snap value: VM_NAME >, sharedLocks= ] >2014-08-12 15:10:47,914 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (http--0.0.0.0-8080-4) [6ea7e4a6] Running command: AddVmFromSnapshotCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-00000000039f Type: VdsGroupsAction group CREATE_VM with role type USER, ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: StorageAction group CREATE_DISK with role type USER, ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group CREATE_VM with role type USER >2014-08-12 15:13:08,561 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (http--0.0.0.0-8080-4) [6ea7e4a6] Locking VM(id = 6bb6b87b-1a5e-4741-826c-b96ee6f84a29) with compensation. >2014-08-12 15:13:08,564 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (http--0.0.0.0-8080-4) [6ea7e4a6] START, SetVmStatusVDSCommand( vmId = 6bb6b87b-1a5e-4741-826c-b96ee6f84a29, status = ImageLocked, exit status = Normal), log id: 4d13bef6 >2014-08-12 15:13:08,568 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (http--0.0.0.0-8080-4) [6ea7e4a6] FINISH, SetVmStatusVDSCommand, log id: 4d13bef6 >2014-08-12 15:13:08,601 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (http--0.0.0.0-8080-4) [6ea7e4a6] Lock freed to object EngineLock [exclusiveLocks= key: de580b19-e975-426f-80de-6c7daf1f24ab value: VM >key: vm_snap value: VM_NAME >, sharedLocks= ] >2014-08-12 15:13:08,639 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (http--0.0.0.0-8080-4) [74a1b05e] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-12 15:13:46,399 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (http--0.0.0.0-8080-4) [74a1b05e] START, CopyImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 016a7ddb-9007-4925-ae72-202649686c94, imageId = 3938bfe7-a921-4b4e-b337-fed4aa931c88, dstImageGroupId = 8d0f851c-38be-4d3e-a278-4af48a16689a, vmId = 6bb6b87b-1a5e-4741-826c-b96ee6f84a29, dstImageId = 2439da84-5f8f-491f-9442-cf49a053e4ca, imageDescription = , dstStorageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, copyVolumeType = LeafVol, volumeFormat = RAW, preallocate = Sparse, postZero = false, force = false), log id: 1ab7341e >2014-08-12 15:13:46,401 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (http--0.0.0.0-8080-4) [74a1b05e] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID >2014-08-12 15:13:46,402 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (http--0.0.0.0-8080-4) [74a1b05e] -- copyImage parameters: > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > spUUID=00000002-0002-0002-0002-00000000037f > vmGUID=6bb6b87b-1a5e-4741-826c-b96ee6f84a29 > srcImageGUID=016a7ddb-9007-4925-ae72-202649686c94 > srcVolUUID=3938bfe7-a921-4b4e-b337-fed4aa931c88 > dstImageGUID=8d0f851c-38be-4d3e-a278-4af48a16689a > dstVolUUID=2439da84-5f8f-491f-9442-cf49a053e4ca > descr= > dstSdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a >2014-08-12 15:13:46,862 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (http--0.0.0.0-8080-4) [74a1b05e] FINISH, CopyImageVDSCommand, return: b6e13ba5-7ce2-4db8-a887-eb032ba759db, log id: 1ab7341e >2014-08-12 15:13:47,034 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (http--0.0.0.0-8080-4) [74a1b05e] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 729324a3-220d-4f24-97c8-dbff6f1b6046 >2014-08-12 15:13:47,034 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (http--0.0.0.0-8080-4) [74a1b05e] CommandMultiAsyncTasks::AttachTask: Attaching task b6e13ba5-7ce2-4db8-a887-eb032ba759db to command 729324a3-220d-4f24-97c8-dbff6f1b6046. >2014-08-12 15:13:53,727 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (http--0.0.0.0-8080-4) [74a1b05e] Adding task b6e13ba5-7ce2-4db8-a887-eb032ba759db (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-12 15:13:54,257 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [74a1b05e] Correlation ID: 6ea7e4a6, Job ID: a34d29e7-80be-4a56-925c-470f7b4873a6, Call Stack: null, Custom Event ID: -1, Message: VM vm_snap creation was initiated by admin. >2014-08-12 15:13:54,258 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (http--0.0.0.0-8080-4) [74a1b05e] BaseAsyncTask::startPollingTask: Starting to poll task b6e13ba5-7ce2-4db8-a887-eb032ba759db. >2014-08-12 15:14:03,643 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-44) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-12 15:14:03,648 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-44) SPMAsyncTask::PollTask: Polling task b6e13ba5-7ce2-4db8-a887-eb032ba759db (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-12 15:14:03,649 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-44) Finished polling Tasks, will poll again in 10 seconds. >2014-08-12 15:15:37,583 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-21) [34d78858] Running command: StopVmCommand internal: false. Entities affected : ID: 7b4eb2dc-f23c-4c2a-8302-f0e1184470d4 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-12 15:15:37,586 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-21) [34d78858] START, DestroyVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, force=false, secondsToWait=0, gracefully=false, reason=), log id: 66475ccb >2014-08-12 15:15:37,606 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-21) [34d78858] START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=7b4eb2dc-f23c-4c2a-8302-f0e1184470d4, force=false, secondsToWait=0, gracefully=false, reason=), log id: 15230040 >2014-08-12 15:15:37,872 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-21) [34d78858] FINISH, DestroyVDSCommand, log id: 15230040 >2014-08-12 15:15:37,881 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-21) [34d78858] FINISH, DestroyVmVDSCommand, return: Down, log id: 66475ccb >2014-08-12 15:15:37,896 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-37) [604e538b] Running command: ProcessDownVmCommand internal: true. >2014-08-12 15:15:37,898 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-21) [34d78858] Correlation ID: 34d78858, Job ID: 262c5006-dcd0-4569-886a-e2115519fa67, Call Stack: null, Custom Event ID: -1, Message: VM vm_rhel_7 powered off by admin (Host: h1) (Reason: Not Specified). >2014-08-12 15:15:45,377 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-39) [23f387e4] Running command: StopVmCommand internal: false. Entities affected : ID: 14c0fa91-1814-438b-8d7a-4e7de1e1da25 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-12 15:15:45,444 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-12 15:15:45,493 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = d8bc9f17-167d-4311-9d78-d65c92bb8506, postZeros = false, forceDelete = false), log id: 19669daa >2014-08-12 15:15:53,212 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-50) [64aff4bd] Running command: StopVmCommand internal: false. Entities affected : ID: 2e77d03b-978a-41d9-a9e9-a43c7035366b Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-12 15:15:53,215 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-50) [64aff4bd] START, DestroyVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b, force=false, secondsToWait=0, gracefully=false, reason=), log id: 57851613 >2014-08-12 15:15:53,241 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-50) [64aff4bd] START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=2e77d03b-978a-41d9-a9e9-a43c7035366b, force=false, secondsToWait=0, gracefully=false, reason=), log id: 57d0a0cb >2014-08-12 15:15:53,498 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-50) [64aff4bd] FINISH, DestroyVDSCommand, log id: 57d0a0cb >2014-08-12 15:15:53,511 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-50) [64aff4bd] FINISH, DestroyVmVDSCommand, return: Down, log id: 57851613 >2014-08-12 15:15:53,513 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-2) [e8be7a1] Running command: ProcessDownVmCommand internal: true. >2014-08-12 15:15:53,552 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-50) [64aff4bd] Correlation ID: 64aff4bd, Job ID: 891c88ee-0846-468a-b242-060d6cf5746e, Call Stack: null, Custom Event ID: -1, Message: VM vm111 powered off by admin (Host: h1) (Reason: Not Specified). >2014-08-12 15:16:05,002 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] FINISH, DeleteImageGroupVDSCommand, log id: 19669daa >2014-08-12 15:16:05,062 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 7f3a34b9-0efb-40ad-9d34-f3fa485c32ec >2014-08-12 15:16:05,062 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] CommandMultiAsyncTasks::AttachTask: Attaching task e67106fb-a328-44a3-9c19-389d65a8379f to command 7f3a34b9-0efb-40ad-9d34-f3fa485c32ec. >2014-08-12 15:16:05,140 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] Adding task e67106fb-a328-44a3-9c19-389d65a8379f (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-12 15:16:05,165 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = ee881bce-a12e-484e-afc4-50a7dfbf6a17, postZeros = false, forceDelete = false), log id: 288759ae >2014-08-12 15:16:11,676 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-28) [128dad3b] Running command: StopVmCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-12 15:16:11,679 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-28) [128dad3b] START, DestroyVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, force=false, secondsToWait=0, gracefully=false, reason=), log id: 495d1032 >2014-08-12 15:16:11,686 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-28) [128dad3b] START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, force=false, secondsToWait=0, gracefully=false, reason=), log id: 48bbef55 >2014-08-12 15:16:11,992 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-28) [128dad3b] FINISH, DestroyVDSCommand, log id: 48bbef55 >2014-08-12 15:16:11,999 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-28) [128dad3b] FINISH, DestroyVmVDSCommand, return: Down, log id: 495d1032 >2014-08-12 15:16:12,006 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-32) [4a3e84e8] Running command: ProcessDownVmCommand internal: true. >2014-08-12 15:16:12,016 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-28) [128dad3b] Correlation ID: 128dad3b, Job ID: 88304a9a-e96b-4d5a-b4cd-84cd26a57d57, Call Stack: null, Custom Event ID: -1, Message: VM vm4 powered off by admin (Host: h1) (Reason: Not Specified). >2014-08-12 15:16:15,008 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-38) Polling and updating Async Tasks: 2 tasks, 1 tasks to poll now >2014-08-12 15:16:35,469 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] FINISH, DeleteImageGroupVDSCommand, log id: 288759ae >2014-08-12 15:16:35,516 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] CommandMultiAsyncTasks::AttachTask: Attaching task a3efca47-2c46-412b-8417-c179e742e88f to command 7f3a34b9-0efb-40ad-9d34-f3fa485c32ec. >2014-08-12 15:16:35,595 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-38) Finished polling Tasks, will poll again in 10 seconds. >2014-08-12 15:16:35,596 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] Adding task a3efca47-2c46-412b-8417-c179e742e88f (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-12 15:16:35,655 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] Correlation ID: 23f387e4, Job ID: 82c72d94-1361-4aa8-8316-1e0bfba4de61, Call Stack: null, Custom Event ID: -1, Message: Suspended VM vm has had its save state cleared by admin (Reason: Not Specified). >2014-08-12 15:16:35,656 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] BaseAsyncTask::startPollingTask: Starting to poll task e67106fb-a328-44a3-9c19-389d65a8379f. >2014-08-12 15:16:35,656 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-39) [5ffd7e89] BaseAsyncTask::startPollingTask: Starting to poll task a3efca47-2c46-412b-8417-c179e742e88f. >2014-08-12 15:16:45,597 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-13) Polling and updating Async Tasks: 3 tasks, 3 tasks to poll now >2014-08-12 15:16:45,791 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-13) SPMAsyncTask::PollTask: Polling task a3efca47-2c46-412b-8417-c179e742e88f (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-12 15:16:45,802 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-13) BaseAsyncTask::onTaskEndSuccess: Task a3efca47-2c46-412b-8417-c179e742e88f (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-12 15:16:45,802 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-13) Task with DB Task ID 346f1848-edbc-4da9-ad99-e912e122ab37 and VDSM Task ID e67106fb-a328-44a3-9c19-389d65a8379f is in state Polling. End action for command 7f3a34b9-0efb-40ad-9d34-f3fa485c32ec will proceed when all the entitys tasks are completed. >2014-08-12 15:16:45,803 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-13) SPMAsyncTask::PollTask: Polling task e67106fb-a328-44a3-9c19-389d65a8379f (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-12 15:16:45,810 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-13) BaseAsyncTask::onTaskEndSuccess: Task e67106fb-a328-44a3-9c19-389d65a8379f (Parent Command StopVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-12 15:16:45,810 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-13) CommandAsyncTask::endActionIfNecessary: All tasks of command 7f3a34b9-0efb-40ad-9d34-f3fa485c32ec has ended -> executing endAction >2014-08-12 15:16:45,811 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-13) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 7f3a34b9-0efb-40ad-9d34-f3fa485c32ec): calling endAction . >2014-08-12 15:16:45,811 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-13) Finished polling Tasks, will poll again in 10 seconds. >2014-08-12 15:16:45,812 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-27) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction StopVm, executionIndex: 0 >2014-08-12 15:16:45,827 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-27) Ending command successfully: org.ovirt.engine.core.bll.StopVmCommand >2014-08-12 15:16:45,860 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-27) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type StopVm completed, handling the result. >2014-08-12 15:16:45,860 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-27) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type StopVm succeeded, clearing tasks. >2014-08-12 15:16:45,868 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-27) SPMAsyncTask::ClearAsyncTask: Attempting to clear task a3efca47-2c46-412b-8417-c179e742e88f >2014-08-12 15:16:45,869 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = a3efca47-2c46-412b-8417-c179e742e88f), log id: 5a6ce81b >2014-08-12 15:16:45,870 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=a3efca47-2c46-412b-8417-c179e742e88f), log id: 23fa72fe >2014-08-12 15:16:46,249 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) FINISH, HSMClearTaskVDSCommand, log id: 23fa72fe >2014-08-12 15:16:46,250 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) FINISH, SPMClearTaskVDSCommand, log id: 5a6ce81b >2014-08-12 15:16:46,265 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-27) BaseAsyncTask::removeTaskFromDB: Removed task a3efca47-2c46-412b-8417-c179e742e88f from DataBase >2014-08-12 15:16:46,266 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-27) SPMAsyncTask::ClearAsyncTask: Attempting to clear task e67106fb-a328-44a3-9c19-389d65a8379f >2014-08-12 15:16:46,267 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = e67106fb-a328-44a3-9c19-389d65a8379f), log id: 59384f50 >2014-08-12 15:16:46,269 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=e67106fb-a328-44a3-9c19-389d65a8379f), log id: 7fdf3af2 >2014-08-12 15:16:47,322 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) FINISH, HSMClearTaskVDSCommand, log id: 7fdf3af2 >2014-08-12 15:16:47,323 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) FINISH, SPMClearTaskVDSCommand, log id: 59384f50 >2014-08-12 15:16:47,375 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-27) BaseAsyncTask::removeTaskFromDB: Removed task e67106fb-a328-44a3-9c19-389d65a8379f from DataBase >2014-08-12 15:16:47,376 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-27) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 7f3a34b9-0efb-40ad-9d34-f3fa485c32ec >2014-08-12 15:18:12,078 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-56) Setting new tasks map. The map contains now 1 tasks >2014-08-12 15:18:13,873 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-44) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-12 15:18:15,808 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-44) Finished polling Tasks, will poll again in 10 seconds. >2014-08-12 15:41:50,574 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-80) SPMAsyncTask::PollTask: Polling task b6e13ba5-7ce2-4db8-a887-eb032ba759db (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-12 15:41:50,593 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-80) BaseAsyncTask::onTaskEndSuccess: Task b6e13ba5-7ce2-4db8-a887-eb032ba759db (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-12 15:41:50,595 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-80) CommandAsyncTask::endActionIfNecessary: All tasks of command 729324a3-220d-4f24-97c8-dbff6f1b6046 has ended -> executing endAction >2014-08-12 15:41:50,596 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-80) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 729324a3-220d-4f24-97c8-dbff6f1b6046): calling endAction . >2014-08-12 15:41:50,597 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVmFromSnapshot, executionIndex: 0 >2014-08-12 15:41:50,626 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (org.ovirt.thread.pool-8-thread-32) [6ea7e4a6] Ending command successfully: org.ovirt.engine.core.bll.AddVmFromSnapshotCommand >2014-08-12 15:41:50,651 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-32) [74a1b05e] Ending command successfully: org.ovirt.engine.core.bll.CopyImageGroupCommand >2014-08-12 15:41:50,659 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-32) [74a1b05e] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 8d0f851c-38be-4d3e-a278-4af48a16689a, imageId = 2439da84-5f8f-491f-9442-cf49a053e4ca), log id: 1c95b9ff >2014-08-12 15:41:50,688 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-32) [74a1b05e] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@a9f6878b, log id: 1c95b9ff >2014-08-12 15:41:50,692 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-8-thread-32) START, SetVmStatusVDSCommand( vmId = 6bb6b87b-1a5e-4741-826c-b96ee6f84a29, status = Down, exit status = Normal), log id: 2d766783 >2014-08-12 15:41:50,698 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-8-thread-32) FINISH, SetVmStatusVDSCommand, log id: 2d766783 >2014-08-12 15:41:50,760 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-32) Correlation ID: 6ea7e4a6, Call Stack: null, Custom Event ID: -1, Message: VM vm_snap creation has been completed. >2014-08-12 15:41:50,760 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVmFromSnapshot completed, handling the result. >2014-08-12 15:41:50,761 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVmFromSnapshot succeeded, clearing tasks. >2014-08-12 15:41:50,768 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-32) SPMAsyncTask::ClearAsyncTask: Attempting to clear task b6e13ba5-7ce2-4db8-a887-eb032ba759db >2014-08-12 15:41:50,768 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-32) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = b6e13ba5-7ce2-4db8-a887-eb032ba759db), log id: 7818dc2f >2014-08-12 15:41:50,770 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-32) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=b6e13ba5-7ce2-4db8-a887-eb032ba759db), log id: 281fcf42 >2014-08-12 15:41:50,847 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-32) FINISH, HSMClearTaskVDSCommand, log id: 281fcf42 >2014-08-12 15:41:50,848 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-32) FINISH, SPMClearTaskVDSCommand, log id: 7818dc2f >2014-08-12 15:41:50,867 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-32) BaseAsyncTask::removeTaskFromDB: Removed task b6e13ba5-7ce2-4db8-a887-eb032ba759db from DataBase >2014-08-12 15:41:50,868 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 729324a3-220d-4f24-97c8-dbff6f1b6046 >2014-08-12 15:43:12,166 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-35) Setting new tasks map. The map contains now 0 tasks >2014-08-12 15:43:12,167 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-35) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-12 15:47:07,686 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-4) [172742c4] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-12 15:47:07,718 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [172742c4] START, IsVmDuringInitiatingVDSCommand( vmId = 5b1ee280-4251-489a-899c-8e9dac73c327), log id: 38f3f697 >2014-08-12 15:47:07,718 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [172742c4] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 38f3f697 >2014-08-12 15:47:07,796 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-17) [172742c4] Running command: RunVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-12 15:47:07,845 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-17) [172742c4] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = ISCSI, connectionList = [{ id: bd327084-78e4-4b84-9a74-9cea2d43c971, connection: 10.35.16.25, iqn: iqn.2011-05.com.redhat:achub-iscsi, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 65a109ed >2014-08-12 15:47:08,750 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-17) [172742c4] FINISH, ConnectStorageServerVDSCommand, return: {bd327084-78e4-4b84-9a74-9cea2d43c971=0}, log id: 65a109ed >2014-08-12 15:47:08,755 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-17) [172742c4] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@5706ab47), log id: 32b3ee97 >2014-08-12 15:47:08,814 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-17) [172742c4] FINISH, UpdateVmDynamicDataVDSCommand, log id: 32b3ee97 >2014-08-12 15:47:08,825 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-17) [172742c4] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, vm=VM [vm3]), log id: 26392e5 >2014-08-12 15:47:08,829 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-17) [172742c4] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, vm=VM [vm3]), log id: 1716aa >2014-08-12 15:47:08,848 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-17) [172742c4] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,timeOffset=0,transparentHugePages=true,vmId=5b1ee280-4251-489a-899c-8e9dac73c327,acpiEnable=true,custom={device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_6c69f8a7-e423-49a5-a4d8-064d8d397d51device_2e3e60b3-5dcc-4990-aa14-c92b63380dbbdevice_482577bb-dce9-43f3-99c9-268d289f6e97=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=482577bb-dce9-43f3-99c9-268d289f6e97, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_6c69f8a7-e423-49a5-a4d8-064d8d397d51device_2e3e60b3-5dcc-4990-aa14-c92b63380dbb=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=2e3e60b3-5dcc-4990-aa14-c92b63380dbb, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_6c69f8a7-e423-49a5-a4d8-064d8d397d51=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=6c69f8a7-e423-49a5-a4d8-064d8d397d51, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_96c8ccf7-eae6-4354-ac32-442f636eee8c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=vnc,smartcardEnable=false,bootMenuEnable=false,tabletEnable=true,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={vram=32768, heads=1}, device=cirrus, type=video, deviceId=2473b074-1cf7-46bc-b515-4e4036653830}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=f76be01f-4e52-4d67-8e1b-0856337e9929}, {shared=false, index=0, volumeID=26308abd-8540-410e-8018-58cd2157084d, propagateErrors=off, format=cow, type=disk, iface=virtio, bootOrder=1, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, imageID=b5576e45-7f13-465f-b26f-05d52faf28d1, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=b5576e45-7f13-465f-b26f-05d52faf28d1}, {shared=false, iface=virtio, GUID=3600144f09dbd050000004e1994c60005, address={bus=0x00, domain=0x0000, slot=0x07, type=pci, function=0x0}, specParams={}, optional=false, propagateErrors=off, device=lun, format=raw, type=disk, readonly=false, deviceId=bdca3776-6597-4323-84f1-4ea75f7d269d}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:53, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=68e93502-4ad4-42a0-9a23-dfa3f67090f5}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=403887de-d0b8-4108-8b43-b4b5d5413859}, {address={bus=0x00, domain=0x0000, slot=0x05, type=pci, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c}],vmName=vm3,cpuType=SandyBridge >2014-08-12 15:47:08,878 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-17) [172742c4] FINISH, CreateVDSCommand, log id: 1716aa >2014-08-12 15:47:08,897 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-17) [172742c4] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 26392e5 >2014-08-12 15:47:08,898 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-17) [172742c4] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-12 15:47:08,906 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-17) [172742c4] Correlation ID: 172742c4, Job ID: ff8045c0-212c-405f-b870-2aadde3d1e44, Call Stack: null, Custom Event ID: -1, Message: VM vm3 was started by admin (Host: h1). >2014-08-12 15:47:12,456 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from WaitForLaunch --> PoweringUp >2014-08-12 15:47:12,457 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-9) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: cc9d97d >2014-08-12 15:47:12,468 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-9) FINISH, FullListVdsCommand, return: [{displaySecurePort=-1, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.35.0.105, pauseCode=NOERR, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, timeOffset=0, transparentHugePages=true, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, acpiEnable=true, custom={device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_6c69f8a7-e423-49a5-a4d8-064d8d397d51device_2e3e60b3-5dcc-4990-aa14-c92b63380dbbdevice_482577bb-dce9-43f3-99c9-268d289f6e97=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=482577bb-dce9-43f3-99c9-268d289f6e97, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_6c69f8a7-e423-49a5-a4d8-064d8d397d51device_2e3e60b3-5dcc-4990-aa14-c92b63380dbb=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=2e3e60b3-5dcc-4990-aa14-c92b63380dbb, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_96c8ccf7-eae6-4354-ac32-442f636eee8c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_6c69f8a7-e423-49a5-a4d8-064d8d397d51=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=6c69f8a7-e423-49a5-a4d8-064d8d397d51, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=5900, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=vnc, pid=16644, smartcardEnable=false, bootMenuEnable=false, tabletEnable=true, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@47185bb9, vmName=vm3, cpuType=SandyBridge}], log id: cc9d97d >2014-08-12 15:47:12,475 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Received a memballoon Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-08-12 15:47:12,475 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Received a vnc Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {port=5900, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, displayIp=10.35.0.105}, device=vnc, type=graphics} >2014-08-12 15:47:16,435 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (http--0.0.0.0-8080-4) [440f12b6] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group CONNECT_TO_VM with role type USER >2014-08-12 15:47:16,474 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-4) [440f12b6] START, SetVmTicketVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, ticket=iu7E+/Nx+Gfg, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 36a16e5e >2014-08-12 15:47:16,485 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-4) [440f12b6] FINISH, SetVmTicketVDSCommand, log id: 36a16e5e >2014-08-12 15:47:16,592 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [440f12b6] Correlation ID: 440f12b6, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM vm3 >2014-08-12 15:47:21,827 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-25) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin is connected to VM vm3. >2014-08-12 15:47:43,411 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin got disconnected from VM vm3. >2014-08-12 15:48:11,139 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from PoweringUp --> Up >2014-08-12 15:48:11,207 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-14) Correlation ID: 172742c4, Job ID: ff8045c0-212c-405f-b870-2aadde3d1e44, Call Stack: null, Custom Event ID: -1, Message: VM vm3 started on Host h1 >2014-08-12 15:58:58,812 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-94) Principal admin::internal synchronized >2014-08-12 15:59:03,573 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-77) Attempting to update VMs/Templates Ovf. >2014-08-12 15:59:03,765 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-77) [7796efc1] Lock Acquired to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-12 15:59:03,769 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-77) [7796efc1] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-12 15:59:03,790 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [7796efc1] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4c16b586-1b20-4f3a-9001-43521e3bb552, imageId = 655fad6a-3526-407e-a7d9-2d5e09102565), log id: 1290b15b >2014-08-12 15:59:03,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [7796efc1] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-12 15:59:03,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [7796efc1] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4c16b586-1b20-4f3a-9001-43521e3bb552 > volUUID=655fad6a-3526-407e-a7d9-2d5e09102565 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Sun Aug 03 11:23:41 IDT 2014"} > >2014-08-12 15:59:03,943 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [7796efc1] FINISH, SetVolumeDescriptionVDSCommand, log id: 1290b15b >2014-08-12 15:59:03,998 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] Running command: UploadStreamCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-12 15:59:04,000 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 8ad6d4f >2014-08-12 15:59:04,001 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] -- executeVdsBrokerCommand: >2014-08-12 15:59:04,001 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGUID=4c16b586-1b20-4f3a-9001-43521e3bb552 > volUUID=655fad6a-3526-407e-a7d9-2d5e09102565 > size=51,200 > >2014-08-12 15:59:04,841 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] FINISH, UploadStreamVDSCommand, log id: 8ad6d4f >2014-08-12 15:59:04,843 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-77) [2d199853] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command ec80a745-a615-44d4-a534-fe7b1d867425 >2014-08-12 15:59:04,843 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-77) [2d199853] CommandMultiAsyncTasks::AttachTask: Attaching task f0daaf32-3bb7-490c-9903-df64262c5aaf to command ec80a745-a615-44d4-a534-fe7b1d867425. >2014-08-12 15:59:04,909 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-77) [2d199853] Adding task f0daaf32-3bb7-490c-9903-df64262c5aaf (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-12 15:59:04,959 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-12 15:59:04,964 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4c16b586-1b20-4f3a-9001-43521e3bb552, imageId = 655fad6a-3526-407e-a7d9-2d5e09102565), log id: 4e5e5191 >2014-08-12 15:59:04,965 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-12 15:59:04,965 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4c16b586-1b20-4f3a-9001-43521e3bb552 > volUUID=655fad6a-3526-407e-a7d9-2d5e09102565 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Tue Aug 12 15:59:03 IDT 2014","Size":51200} > >2014-08-12 15:59:05,091 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] FINISH, SetVolumeDescriptionVDSCommand, log id: 4e5e5191 >2014-08-12 15:59:05,101 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4cab3024-3f6d-435a-bf22-a6e7daac3bd9, imageId = 612a0f89-926c-48c5-9804-b258314e4838), log id: 43b725a4 >2014-08-12 15:59:05,102 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-12 15:59:05,102 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4cab3024-3f6d-435a-bf22-a6e7daac3bd9 > volUUID=612a0f89-926c-48c5-9804-b258314e4838 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Sun Aug 03 11:23:41 IDT 2014"} > >2014-08-12 15:59:05,224 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [2d199853] FINISH, SetVolumeDescriptionVDSCommand, log id: 43b725a4 >2014-08-12 15:59:05,265 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-77) [50cc40dc] Running command: UploadStreamCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-12 15:59:05,267 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-77) [50cc40dc] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 19fcc13d >2014-08-12 15:59:05,268 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-77) [50cc40dc] -- executeVdsBrokerCommand: >2014-08-12 15:59:05,268 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-77) [50cc40dc] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGUID=4cab3024-3f6d-435a-bf22-a6e7daac3bd9 > volUUID=612a0f89-926c-48c5-9804-b258314e4838 > size=51,200 > >2014-08-12 15:59:06,429 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-77) [50cc40dc] FINISH, UploadStreamVDSCommand, log id: 19fcc13d >2014-08-12 15:59:06,432 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-77) [50cc40dc] CommandMultiAsyncTasks::AttachTask: Attaching task 526ff8c7-9348-4e58-b292-1e436a5f7a61 to command ec80a745-a615-44d4-a534-fe7b1d867425. >2014-08-12 15:59:06,491 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-77) [50cc40dc] Adding task 526ff8c7-9348-4e58-b292-1e436a5f7a61 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-12 15:59:06,508 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-77) [50cc40dc] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-12 15:59:06,509 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-77) [50cc40dc] Trying to release exclusive lock which does not exist, lock key: d54e6e6c-81f2-41f2-90ab-7e3c2457938aSTORAGE >2014-08-12 15:59:06,509 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-77) [50cc40dc] Trying to release exclusive lock which does not exist, lock key: 4c16b586-1b20-4f3a-9001-43521e3bb552DISK >2014-08-12 15:59:06,510 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-77) [50cc40dc] Trying to release exclusive lock which does not exist, lock key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9DISK >2014-08-12 15:59:06,511 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-77) [50cc40dc] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-12 15:59:06,513 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [50cc40dc] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4cab3024-3f6d-435a-bf22-a6e7daac3bd9, imageId = 612a0f89-926c-48c5-9804-b258314e4838), log id: 4d33235a >2014-08-12 15:59:06,515 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [50cc40dc] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-12 15:59:06,515 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [50cc40dc] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4cab3024-3f6d-435a-bf22-a6e7daac3bd9 > volUUID=612a0f89-926c-48c5-9804-b258314e4838 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Tue Aug 12 15:59:03 IDT 2014","Size":51200} > >2014-08-12 15:59:06,607 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-77) [50cc40dc] FINISH, SetVolumeDescriptionVDSCommand, log id: 4d33235a >2014-08-12 15:59:06,616 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-77) [50cc40dc] BaseAsyncTask::startPollingTask: Starting to poll task f0daaf32-3bb7-490c-9903-df64262c5aaf. >2014-08-12 15:59:06,617 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-77) [50cc40dc] BaseAsyncTask::startPollingTask: Starting to poll task 526ff8c7-9348-4e58-b292-1e436a5f7a61. >2014-08-12 15:59:10,637 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-3) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-12 15:59:10,712 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) SPMAsyncTask::PollTask: Polling task f0daaf32-3bb7-490c-9903-df64262c5aaf (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-12 15:59:10,713 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) BaseAsyncTask::onTaskEndSuccess: Task f0daaf32-3bb7-490c-9903-df64262c5aaf (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-12 15:59:10,715 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-3) Task with DB Task ID 90419538-7393-4943-9527-e7a9a62d54a6 and VDSM Task ID 526ff8c7-9348-4e58-b292-1e436a5f7a61 is in state Polling. End action for command ec80a745-a615-44d4-a534-fe7b1d867425 will proceed when all the entitys tasks are completed. >2014-08-12 15:59:10,716 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) SPMAsyncTask::PollTask: Polling task 526ff8c7-9348-4e58-b292-1e436a5f7a61 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-12 15:59:10,718 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) BaseAsyncTask::onTaskEndSuccess: Task 526ff8c7-9348-4e58-b292-1e436a5f7a61 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-12 15:59:10,719 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-3) CommandAsyncTask::endActionIfNecessary: All tasks of command ec80a745-a615-44d4-a534-fe7b1d867425 has ended -> executing endAction >2014-08-12 15:59:10,720 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-3) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: ec80a745-a615-44d4-a534-fe7b1d867425): calling endAction . >2014-08-12 15:59:10,721 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-08-12 15:59:10,744 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-50) [7796efc1] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-08-12 15:59:10,745 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-50) [7796efc1] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-12 15:59:10,745 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-50) [7796efc1] Trying to release exclusive lock which does not exist, lock key: d54e6e6c-81f2-41f2-90ab-7e3c2457938aSTORAGE >2014-08-12 15:59:10,746 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-50) [7796efc1] Trying to release exclusive lock which does not exist, lock key: 4c16b586-1b20-4f3a-9001-43521e3bb552DISK >2014-08-12 15:59:10,747 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-50) [7796efc1] Trying to release exclusive lock which does not exist, lock key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9DISK >2014-08-12 15:59:10,748 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-50) [7796efc1] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-12 15:59:10,749 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7796efc1] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-08-12 15:59:10,750 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7796efc1] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-08-12 15:59:10,751 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7796efc1] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 526ff8c7-9348-4e58-b292-1e436a5f7a61 >2014-08-12 15:59:10,752 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7796efc1] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 526ff8c7-9348-4e58-b292-1e436a5f7a61), log id: 6a0164ba >2014-08-12 15:59:10,754 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7796efc1] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=526ff8c7-9348-4e58-b292-1e436a5f7a61), log id: 17c25b5 >2014-08-12 15:59:10,830 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7796efc1] FINISH, HSMClearTaskVDSCommand, log id: 17c25b5 >2014-08-12 15:59:10,831 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7796efc1] FINISH, SPMClearTaskVDSCommand, log id: 6a0164ba >2014-08-12 15:59:10,856 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7796efc1] BaseAsyncTask::removeTaskFromDB: Removed task 526ff8c7-9348-4e58-b292-1e436a5f7a61 from DataBase >2014-08-12 15:59:10,857 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7796efc1] SPMAsyncTask::ClearAsyncTask: Attempting to clear task f0daaf32-3bb7-490c-9903-df64262c5aaf >2014-08-12 15:59:10,859 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7796efc1] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = f0daaf32-3bb7-490c-9903-df64262c5aaf), log id: 30883ca0 >2014-08-12 15:59:10,861 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7796efc1] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=f0daaf32-3bb7-490c-9903-df64262c5aaf), log id: 40444544 >2014-08-12 15:59:10,909 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7796efc1] FINISH, HSMClearTaskVDSCommand, log id: 40444544 >2014-08-12 15:59:10,910 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7796efc1] FINISH, SPMClearTaskVDSCommand, log id: 30883ca0 >2014-08-12 15:59:10,922 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7796efc1] BaseAsyncTask::removeTaskFromDB: Removed task f0daaf32-3bb7-490c-9903-df64262c5aaf from DataBase >2014-08-12 15:59:10,923 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7796efc1] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity ec80a745-a615-44d4-a534-fe7b1d867425 >2014-08-12 16:00:12,175 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) Setting new tasks map. The map contains now 0 tasks >2014-08-12 16:00:12,177 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-12 16:58:58,857 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-31) Principal admin::internal synchronized >2014-08-12 16:59:06,618 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-53) Attempting to update VMs/Templates Ovf. >2014-08-13 06:49:08,662 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-7) [6366e0c6] Start deleteAgedOutAuditLogs >2014-08-13 06:49:08,665 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-18) Attempting to update VMs/Templates Ovf. >2014-08-13 06:49:08,665 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-42) [46abe3f3] Start deleteAgedOutCommandEntities >2014-08-13 06:49:08,988 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-63) Command SpmStatusVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:49:09,000 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-42) [46abe3f3] Finished deleteAgedOutCommandEntities >2014-08-13 06:49:09,004 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-1) [7997d276] Running command: LogoutUserCommand internal: false. >2014-08-13 06:49:09,012 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-63) [7425e061] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 06:49:09,017 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-63) [7425e061] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-13 06:49:09,018 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-63) [7425e061] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-13 06:49:09,025 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-4) [3414625] Running command: LogoutUserCommand internal: false. >2014-08-13 06:49:09,499 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-7) [6366e0c6] Finished deleteAgedOutAuditLogs >2014-08-13 06:49:09,501 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-2) Principal admin::internal synchronized >2014-08-13 06:49:09,500 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [7997d276] Correlation ID: 7997d276, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-13 06:49:09,500 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) [3414625] Correlation ID: 3414625, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-13 06:49:09,714 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-63) [7425e061] Correlation ID: 7425e061, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host h1, Error: Network error during communication with the Host.). >2014-08-13 06:51:28,797 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-95) Command ListVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException >2014-08-13 06:51:28,800 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Host h1 is not responding. It will stay in Connecting state for a grace period of $80 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:51:28,802 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:407) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.fetchRunningVms(VdsUpdateRunTimeInfo.java:971) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:920) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:638) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.util.concurrent.TimeoutException > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:119) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy.invoke(XmlRpcUtils.java:200) [vdsbroker.jar:] > at com.sun.proxy.$Proxy83.list(Unknown Source) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper.list(VdsServerWrapper.java:127) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand.executeVdsBrokerCommand(ListVDSCommand.java:23) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > ... 14 more > >2014-08-13 06:51:28,848 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.util.concurrent.TimeoutException >2014-08-13 06:52:09,731 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) [7425e061] IrsBroker::Failed::GetStoragePoolInfoVDS due to: TimeoutException: >2014-08-13 06:52:27,918 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:52:27,919 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of $80 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:52:27,919 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-58) Command SpmStatusVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:52:27,920 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:52:27,927 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) IrsBroker::Failed::GetStoragePoolInfoVDS due to: UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:52:27,935 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:52:30,960 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:52:30,961 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host h1 is not responding. It will stay in Connecting state for a grace period of $80 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:52:31,010 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) START, SetVmStatusVDSCommand( vmId = 5b1ee280-4251-489a-899c-8e9dac73c327, status = Unknown, exit status = Normal), log id: 50749c09 >2014-08-13 06:52:31,037 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, SetVmStatusVDSCommand, log id: 50749c09 >2014-08-13 06:52:31,087 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm3 was set to the Unknown status. >2014-08-13 06:52:31,087 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Server failed to respond, vds_id = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds_name = h1, vm_count = 1, spm_status = SPM, non-responsive_timeout (seconds) = 80, error = java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:52:31,103 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) >, Custom Event ID: -1, Message: Host h1 is non responsive. >2014-08-13 06:52:31,112 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-8-thread-40) ResourceManager::vdsNotResponding entered for Host 9c67d903-8424-4cda-8bb1-3d8652b92d77, battlestar.tlv.redhat.com >2014-08-13 06:52:31,155 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:52:31,180 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-40) [2690effe] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-13 06:52:31,181 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-40) [2690effe] Opening SSH Soft Fencing session on host battlestar.tlv.redhat.com >2014-08-13 06:52:31,282 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-40) [2690effe] SSH Soft Fencing command failed on host battlestar.tlv.redhat.com: null >Stdout: >Stderr: >Stacktrace: java.nio.channels.UnresolvedAddressException: java.nio.channels.UnresolvedAddressException > at sun.nio.ch.Net.checkAddress(Net.java:127) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.implConnect(UnixAsynchronousSocketChannelImpl.java:301) [rt.jar:1.7.0_25] > at sun.nio.ch.AsynchronousSocketChannelImpl.connect(AsynchronousSocketChannelImpl.java:208) [rt.jar:1.7.0_25] > at org.apache.sshd.common.io.nio2.Nio2Connector.connect(Nio2Connector.java:47) > at org.apache.sshd.SshClient.connect(SshClient.java:334) > at org.apache.sshd.SshClient.connect(SshClient.java:320) > at org.apache.sshd.SshClient.connect(SshClient.java:310) > at org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:284) [uutils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:87) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:53) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1170) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1921) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view8.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$4.run(VdsEventListener.java:240) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:52:31,355 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-40) [3d8065e4] Lock Acquired to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-13 06:52:31,356 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-40) [3d8065e4] Fence failed on vds h1 which is spm of pool 00000002-0002-0002-0002-00000000037f - moving pool to non operational >2014-08-13 06:52:31,361 WARN [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (org.ovirt.thread.pool-8-thread-40) [ebeadfa] The message key SetStoragePoolStatus is missing from bundles/ExecutionMessages >2014-08-13 06:52:31,395 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-8-thread-40) [ebeadfa] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 06:52:31,412 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-40) [ebeadfa] Correlation ID: ebeadfa, Job ID: e8d3c890-9173-4c20-85ef-657d2aa1d2c8, Call Stack: null, Custom Event ID: -1, Message: Fencing failed on Storage Pool Manager h1 for Data Center Default. Setting status to Non-Operational. >2014-08-13 06:52:31,420 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-8-thread-40) [ebeadfa] Lock freed to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-13 06:52:31,420 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-40) [ebeadfa] Failed to run Fence script on vds:h1. >2014-08-13 06:52:31,428 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-40) [ebeadfa] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-08-13 06:52:31,429 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-40) [ebeadfa] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-08-13 06:52:31,429 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-40) [ebeadfa] Trying to release exclusive lock which does not exist, lock key: 9c67d903-8424-4cda-8bb1-3d8652b92d77VDS_FENCE >2014-08-13 06:52:31,430 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-40) [ebeadfa] Lock freed to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-13 06:52:34,185 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:52:34,186 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:52:34,236 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:52:37,268 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:52:37,268 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:52:37,317 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:53:36,419 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:53:36,421 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:53:36,731 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:53:39,797 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:53:39,799 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:53:39,820 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:53:42,844 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:53:42,846 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:53:42,893 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:53:45,924 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:53:45,925 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:53:45,967 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:54:45,036 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:54:45,038 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:54:45,224 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:54:48,258 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:54:48,259 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:54:48,313 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:54:51,368 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:54:51,370 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-37) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:54:51,428 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:54:54,490 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:54:54,492 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:54:54,534 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:54:57,978 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:54:57,978 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:54:58,045 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:55:01,061 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:55:01,062 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:55:01,126 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:55:04,151 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:55:04,153 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:55:04,249 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:55:07,280 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:55:07,282 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:55:07,330 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:55:10,823 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:55:10,824 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:55:10,890 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:55:13,918 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) [50cc40dc] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:55:13,919 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) [50cc40dc] Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:55:13,980 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) [50cc40dc] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:55:17,005 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:55:17,007 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:55:17,102 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:55:20,129 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 06:55:20,131 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 06:55:20,191 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 06:55:24,933 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-62) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 4221c66d >2014-08-13 06:55:25,113 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, GetHardwareInfoVDSCommand, log id: 4221c66d >2014-08-13 06:55:25,129 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-13 06:55:25,130 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is running with disabled SELinux. >2014-08-13 06:55:25,165 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-62) [3e5c0424] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-13 06:55:25,458 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) [3e5c0424] VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from Unknown --> Up >2014-08-13 06:55:25,510 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) [3e5c0424] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm3 status was restored to Up. >2014-08-13 06:55:25,525 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-62) [1a106fe0] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 06:55:25,546 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-62) [5834c2de] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 06:55:25,566 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-62) [5834c2de] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = NFS, connectionList = [{ id: 1670ace3-7db6-4c73-9b3c-614dcc85bd46, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc5, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc4, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 4844a036 >2014-08-13 06:55:25,827 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-62) [5834c2de] FINISH, ConnectStorageServerVDSCommand, return: {1670ace3-7db6-4c73-9b3c-614dcc85bd46=0, 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a=0}, log id: 4844a036 >2014-08-13 06:55:25,828 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-62) [5834c2de] Host h1 storage connection was succeeded >2014-08-13 06:55:25,839 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-32) START, ConnectStoragePoolVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vdsId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, masterVersion = 1), log id: 448764a >2014-08-13 06:55:26,441 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-32) FINISH, ConnectStoragePoolVDSCommand, log id: 448764a >2014-08-13 06:55:27,120 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-62) [5834c2de] START, SetMOMPolicyParametersVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 3dbb08ac >2014-08-13 06:55:27,502 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-62) [5834c2de] FINISH, SetMOMPolicyParametersVDSCommand, log id: 3dbb08ac >2014-08-13 06:55:27,514 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) [5834c2de] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host h1 was set to Up. >2014-08-13 06:55:27,520 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) [5834c2de] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h1. >2014-08-13 06:55:27,551 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-62) [39efaed5] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-13 06:55:29,360 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 42 GB of free space >2014-08-13 06:55:29,385 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 42 GB of free space >2014-08-13 10:02:21,645 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-92) Attempting to update VMs/Templates Ovf. >2014-08-13 10:02:21,664 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-91) [5a42d7bf] Principal admin::internal synchronized >2014-08-13 10:05:17,243 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-69) Command ListVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException >2014-08-13 10:05:17,245 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Host h1 is not responding. It will stay in Connecting state for a grace period of $80 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:05:17,246 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:407) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.fetchRunningVms(VdsUpdateRunTimeInfo.java:971) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:920) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:638) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.util.concurrent.TimeoutException > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:119) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy.invoke(XmlRpcUtils.java:200) [vdsbroker.jar:] > at com.sun.proxy.$Proxy83.list(Unknown Source) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper.list(VdsServerWrapper.java:127) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand.executeVdsBrokerCommand(ListVDSCommand.java:23) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > ... 14 more > >2014-08-13 10:05:17,281 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.util.concurrent.TimeoutException >2014-08-13 10:05:21,711 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-23) Command SpmStatusVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException >2014-08-13 10:05:21,714 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-23) [302eca83] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 10:05:21,717 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [302eca83] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-13 10:05:21,719 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [302eca83] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-13 10:05:21,777 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [302eca83] Correlation ID: 302eca83, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host h1, Error: Network error during communication with the Host.). >2014-08-13 10:05:22,670 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-30) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 31a9a3fc >2014-08-13 10:05:24,073 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-30) FINISH, GetHardwareInfoVDSCommand, log id: 31a9a3fc >2014-08-13 10:05:24,155 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-13 10:05:24,156 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) Host h1 is running with disabled SELinux. >2014-08-13 10:05:24,199 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-30) [3081c6e9] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-13 10:05:24,434 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-30) [79daf6a0] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 10:05:24,446 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-30) [73ff1308] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 10:05:24,456 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-30) [73ff1308] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = NFS, connectionList = [{ id: 1670ace3-7db6-4c73-9b3c-614dcc85bd46, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc5, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc4, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 4076f21e >2014-08-13 10:05:24,678 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-30) [73ff1308] FINISH, ConnectStorageServerVDSCommand, return: {1670ace3-7db6-4c73-9b3c-614dcc85bd46=0, 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a=0}, log id: 4076f21e >2014-08-13 10:05:24,679 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-30) [73ff1308] Host h1 storage connection was succeeded >2014-08-13 10:05:24,686 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-24) START, ConnectStoragePoolVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vdsId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, masterVersion = 1), log id: 530c5fd1 >2014-08-13 10:05:25,357 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-24) FINISH, ConnectStoragePoolVDSCommand, log id: 530c5fd1 >2014-08-13 10:05:25,959 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-30) [73ff1308] START, SetMOMPolicyParametersVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 7a861874 >2014-08-13 10:05:26,219 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-30) [73ff1308] FINISH, SetMOMPolicyParametersVDSCommand, log id: 7a861874 >2014-08-13 10:05:26,258 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) [73ff1308] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host h1 was set to Up. >2014-08-13 10:05:26,263 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) [73ff1308] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h1. >2014-08-13 10:05:26,287 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-30) [5e1ffa0b] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-13 10:08:21,785 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) [302eca83] IrsBroker::Failed::GetStoragePoolInfoVDS due to: TimeoutException: >2014-08-13 10:25:12,174 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetStatsVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException >2014-08-13 10:25:12,179 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h1 is not responding. It will stay in Connecting state for a grace period of $80 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:25:12,181 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) vds::refreshVdsStats Failed getVdsStats, vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, error = VDSNetworkException: java.util.concurrent.TimeoutException >2014-08-13 10:25:12,182 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsStats(VdsUpdateRunTimeInfo.java:669) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:623) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.util.concurrent.TimeoutException > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:119) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy.invoke(XmlRpcUtils.java:200) [vdsbroker.jar:] > at com.sun.proxy.$Proxy83.getVdsStats(Unknown Source) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper.getVdsStats(VdsServerWrapper.java:173) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand.executeVdsBrokerCommand(GetStatsVDSCommand.java:23) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > ... 12 more > >2014-08-13 10:25:12,233 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.util.concurrent.TimeoutException >2014-08-13 10:25:15,716 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-36) Command SpmStatusVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException >2014-08-13 10:25:15,720 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-36) [52d1af7d] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 10:25:15,723 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-36) [52d1af7d] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-13 10:25:15,725 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-36) [52d1af7d] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-13 10:25:15,785 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-36) [52d1af7d] Correlation ID: 52d1af7d, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host h1, Error: Network error during communication with the Host.). >2014-08-13 10:26:11,339 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:26:11,341 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] Host h1 is not responding. It will stay in Connecting state for a grace period of $80 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:26:11,342 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:26:11,365 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) [2aa3e99e] Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:26:14,407 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:26:14,408 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of $80 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:26:14,459 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (DefaultQuartzScheduler_Worker-96) START, SetVmStatusVDSCommand( vmId = 5b1ee280-4251-489a-899c-8e9dac73c327, status = Unknown, exit status = Normal), log id: 422b122c >2014-08-13 10:26:14,769 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (DefaultQuartzScheduler_Worker-96) FINISH, SetVmStatusVDSCommand, log id: 422b122c >2014-08-13 10:26:14,835 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-96) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm3 was set to the Unknown status. >2014-08-13 10:26:14,836 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Server failed to respond, vds_id = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds_name = h1, vm_count = 1, spm_status = SPM, non-responsive_timeout (seconds) = 80, error = java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:26:14,852 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-96) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) >, Custom Event ID: -1, Message: Host h1 is non responsive. >2014-08-13 10:26:14,862 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-8-thread-44) ResourceManager::vdsNotResponding entered for Host 9c67d903-8424-4cda-8bb1-3d8652b92d77, battlestar.tlv.redhat.com >2014-08-13 10:26:14,876 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:26:14,918 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-44) [4c2aa46f] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-13 10:26:14,920 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-44) [4c2aa46f] Opening SSH Soft Fencing session on host battlestar.tlv.redhat.com >2014-08-13 10:26:14,940 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-44) [4c2aa46f] SSH Soft Fencing command failed on host battlestar.tlv.redhat.com: null >Stdout: >Stderr: >Stacktrace: java.nio.channels.UnresolvedAddressException: java.nio.channels.UnresolvedAddressException > at sun.nio.ch.Net.checkAddress(Net.java:127) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.implConnect(UnixAsynchronousSocketChannelImpl.java:301) [rt.jar:1.7.0_25] > at sun.nio.ch.AsynchronousSocketChannelImpl.connect(AsynchronousSocketChannelImpl.java:208) [rt.jar:1.7.0_25] > at org.apache.sshd.common.io.nio2.Nio2Connector.connect(Nio2Connector.java:47) > at org.apache.sshd.SshClient.connect(SshClient.java:334) > at org.apache.sshd.SshClient.connect(SshClient.java:320) > at org.apache.sshd.SshClient.connect(SshClient.java:310) > at org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:284) [uutils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:87) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:53) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1170) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1921) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view8.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$4.run(VdsEventListener.java:240) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:26:15,017 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-44) [38d01791] Lock Acquired to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-13 10:26:15,019 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-44) [38d01791] Fence failed on vds h1 which is spm of pool 00000002-0002-0002-0002-00000000037f - moving pool to non operational >2014-08-13 10:26:15,024 WARN [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (org.ovirt.thread.pool-8-thread-44) [167aa92f] The message key SetStoragePoolStatus is missing from bundles/ExecutionMessages >2014-08-13 10:26:15,059 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-8-thread-44) [167aa92f] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 10:26:15,084 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-44) [167aa92f] Correlation ID: 167aa92f, Job ID: 3b4762cf-ad60-4e62-b5cb-f73892d4d42e, Call Stack: null, Custom Event ID: -1, Message: Fencing failed on Storage Pool Manager h1 for Data Center Default. Setting status to Non-Operational. >2014-08-13 10:26:15,092 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-8-thread-44) [167aa92f] Lock freed to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-13 10:26:15,093 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-44) [167aa92f] Failed to run Fence script on vds:h1. >2014-08-13 10:26:15,097 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-44) [167aa92f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-08-13 10:26:15,097 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-44) [167aa92f] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-08-13 10:26:15,098 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-44) [167aa92f] Trying to release exclusive lock which does not exist, lock key: 9c67d903-8424-4cda-8bb1-3d8652b92d77VDS_FENCE >2014-08-13 10:26:15,098 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-44) [167aa92f] Lock freed to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-13 10:26:17,918 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:26:17,919 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-37) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:26:18,040 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:26:21,078 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:26:21,078 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:26:21,138 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:27:20,222 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:27:20,223 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:27:20,245 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:27:23,276 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:27:23,277 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-29) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:27:23,309 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:27:26,342 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:27:26,343 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:27:26,372 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:27:29,402 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:27:29,403 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:27:29,501 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:27:33,332 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:27:33,333 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:27:33,393 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:27:36,432 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:27:36,433 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:27:36,481 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:27:39,495 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:27:39,496 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:27:39,536 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:27:42,550 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:27:42,551 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:27:42,608 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:27:46,096 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) [5e1ffa0b] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:27:46,096 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) [5e1ffa0b] Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:27:46,110 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) [5e1ffa0b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:27:49,123 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:27:49,124 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:27:49,149 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:27:52,163 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:27:52,164 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:27:52,187 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:27:55,211 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:27:55,212 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:27:55,259 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor395.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:29:27,507 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-13 10:29:27,508 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-13 10:29:27,512 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-13 10:29:27,515 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-13 10:29:27,518 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-13 10:29:27,522 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-13 10:29:27,528 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-13 10:29:27,529 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-13 10:29:27,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-13 10:29:27,534 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-13 10:29:27,535 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-13 10:29:27,539 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-13 10:29:27,543 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-13 10:29:27,544 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-13 10:29:27,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-13 10:29:27,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-13 10:29:27,556 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-13 10:29:27,556 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-13 10:29:27,557 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-13 10:29:27,557 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-13 10:29:27,557 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-13 10:29:27,558 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-13 10:29:27,560 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DB_USER" is "engine". >2014-08-13 10:29:27,560 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-13 10:29:27,561 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-13 10:29:27,565 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-13 10:29:27,565 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-13 10:29:27,566 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-13 10:29:27,570 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-13 10:29:27,570 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-13 10:29:27,571 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-13 10:29:27,572 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-13 10:29:27,575 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-13 10:29:27,578 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-13 10:29:27,579 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-13 10:29:27,580 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-13 10:29:27,583 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-13 10:29:27,585 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-13 10:29:27,589 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-13 10:29:27,589 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-13 10:29:27,593 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-13 10:29:27,593 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-13 10:29:27,594 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-13 10:29:27,596 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-13 10:29:27,599 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-13 10:29:27,600 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-13 10:29:27,602 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-13 10:29:27,603 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-13 10:29:27,604 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-13 10:29:27,604 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-13 10:29:27,605 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-13 10:29:27,605 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-13 10:29:27,606 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-13 10:29:27,606 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-13 10:29:27,607 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-13 10:29:27,607 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-13 10:29:27,608 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-13 10:29:27,608 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-13 10:29:27,610 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-13 10:29:27,611 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-13 10:29:27,611 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-13 10:29:27,612 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-13 10:29:27,613 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-13 10:29:27,614 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_USER" is "ovirt". >2014-08-13 10:29:27,614 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-13 10:29:27,615 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-13 10:29:27,615 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-13 10:29:27,616 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-13 10:29:27,616 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-13 10:29:27,617 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-4) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-13 10:29:28,030 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) Start initializing Backend >2014-08-13 10:29:28,171 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-5) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:29:28,295 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-5) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:29:28,357 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-5) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-13 10:29:28,368 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-5) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-13 10:29:28,478 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) Running ovirt-engine 3.5.0.0 >2014-08-13 10:29:28,479 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-5) Start initializing dictionaries >2014-08-13 10:29:28,481 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-5) Finished initializing dictionaries >2014-08-13 10:29:28,482 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Start initializing AuditLogCleanupManager >2014-08-13 10:29:28,483 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-13 10:29:28,509 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Finished initializing AuditLogCleanupManager >2014-08-13 10:29:28,510 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Start initializing CommandEntityCleanupManager >2014-08-13 10:29:28,510 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-13 10:29:28,512 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Finished initializing CommandEntityCleanupManager >2014-08-13 10:29:28,514 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-5) Start initializing TagsDirector >2014-08-13 10:29:28,527 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-5) Tag root added to tree >2014-08-13 10:29:28,535 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-5) Finished initializing TagsDirector >2014-08-13 10:29:28,536 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-5) Start initializing IsoDomainListSyncronizer >2014-08-13 10:29:28,542 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-5) Finished initializing IsoDomainListSyncronizer >2014-08-13 10:29:28,613 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) Completed initializing handlers >2014-08-13 10:29:28,637 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-5) Start initializing ErrorTranslatorImpl >2014-08-13 10:29:28,647 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-5) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-13 10:29:28,649 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-5) Finished initializing ErrorTranslatorImpl >2014-08-13 10:29:28,649 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-5) Start initializing ErrorTranslatorImpl >2014-08-13 10:29:28,650 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-5) Finished initializing ErrorTranslatorImpl >2014-08-13 10:29:28,650 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) Mark incomplete jobs as UNKNOWN >2014-08-13 10:29:28,697 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-5) Start initializing JobRepositoryCleanupManager >2014-08-13 10:29:28,697 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-5) Finished initializing JobRepositoryCleanupManager >2014-08-13 10:29:28,698 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-5) Start initializing AutoRecoveryManager >2014-08-13 10:29:28,699 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-5) Finished initializing AutoRecoveryManager >2014-08-13 10:29:28,700 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-5) Start initializing ExecutionMessageDirector >2014-08-13 10:29:28,704 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-5) Finished initializing ExecutionMessageDirector >2014-08-13 10:29:28,755 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-5) Start of enabled extensions list >2014-08-13 10:29:28,755 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-5) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-13 10:29:28,756 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-5) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-13 10:29:28,757 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-5) End of enabled extensions list >2014-08-13 10:29:28,827 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-1) Start initializing DbUserCacheManager >2014-08-13 10:29:28,832 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-1) Finished initializing DbUserCacheManager >2014-08-13 10:29:28,875 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-13 10:29:28,880 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-1) Initialization of AsyncTaskManager completed successfully. >2014-08-13 10:29:28,881 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-1) Start initializing ResourceManager >2014-08-13 10:29:28,985 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-13 10:29:28,989 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: UNASSIGNED not exist in string table >2014-08-13 10:29:28,990 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-13 10:29:28,990 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-13 10:29:28,991 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-13 10:29:28,991 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-13 10:29:28,992 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-13 10:29:28,992 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-13 10:29:28,993 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-13 10:29:28,994 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-13 10:29:28,995 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-13 10:29:28,995 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-13 10:29:28,996 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: DWH_STOPPED not exist in string table >2014-08-13 10:29:28,996 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: DWH_STARTED not exist in string table >2014-08-13 10:29:28,997 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: DWH_ERROR not exist in string table >2014-08-13 10:29:28,998 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-13 10:29:28,998 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-13 10:29:28,999 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-13 10:29:28,999 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-13 10:29:29,105 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm3 was set to the Unknown status. >2014-08-13 10:29:29,129 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-1) Entered VdsManager constructor >2014-08-13 10:29:29,148 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-1) Initialize vdsBroker (galactica,54,321) >2014-08-13 10:29:29,186 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-1) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-13 10:29:29,187 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-1) Entered VdsManager constructor >2014-08-13 10:29:29,198 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-1) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-13 10:29:29,281 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-1) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-13 10:29:29,307 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-1) Finished initializing ResourceManager >2014-08-13 10:29:29,314 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-1) Initialization of OvfDataUpdater completed successfully. >2014-08-13 10:29:29,315 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-1) Start scheduling to enable vds load balancer >2014-08-13 10:29:29,316 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-1) Finished scheduling to enable vds load balancer >2014-08-13 10:29:29,316 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-1) Start HA Reservation check >2014-08-13 10:29:29,317 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-1) Finished HA Reservation check >2014-08-13 10:29:29,326 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-1) Init VM custom properties utilities >2014-08-13 10:29:29,327 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-1) Init device custom properties utilities >2014-08-13 10:29:29,331 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-1) Initializing Scheduling manager >2014-08-13 10:29:29,366 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-1) External scheduler disabled, discovery skipped >2014-08-13 10:29:29,366 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-1) Initialized Scheduling manager >2014-08-13 10:29:29,367 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-1) Initializing DWH Heart Beat >2014-08-13 10:29:29,366 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-13 10:29:29,369 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-1) DWH Heart Beat initialized >2014-08-13 10:29:29,373 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-13 10:29:29,374 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-13 10:29:32,203 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h2 >2014-08-13 10:29:32,641 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:29:32,642 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of $80 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:29:32,697 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:29:32,751 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:29:35,788 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:29:35,789 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Host h1 is not responding. It will stay in Connecting state for a grace period of $80 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:29:35,789 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:29:35,805 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:29:38,850 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:29:38,851 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Host h1 is not responding. It will stay in Connecting state for a grace period of $80 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:29:38,918 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (DefaultQuartzScheduler_Worker-20) START, SetVmStatusVDSCommand( vmId = 5b1ee280-4251-489a-899c-8e9dac73c327, status = Unknown, exit status = Normal), log id: 1f648418 >2014-08-13 10:29:38,967 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, SetVmStatusVDSCommand, log id: 1f648418 >2014-08-13 10:29:39,058 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm3 was set to the Unknown status. >2014-08-13 10:29:39,059 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Server failed to respond, vds_id = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds_name = h1, vm_count = 1, spm_status = SPM, non-responsive_timeout (seconds) = 80, error = java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:29:39,091 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) >, Custom Event ID: -1, Message: Host h1 is non responsive. >2014-08-13 10:29:39,111 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-8-thread-7) ResourceManager::vdsNotResponding entered for Host 9c67d903-8424-4cda-8bb1-3d8652b92d77, battlestar.tlv.redhat.com >2014-08-13 10:29:39,133 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:29:39,308 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-7) [31a1b8bb] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-13 10:29:39,352 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-7) [31a1b8bb] Opening SSH Soft Fencing session on host battlestar.tlv.redhat.com >2014-08-13 10:29:39,440 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-7) [31a1b8bb] SSH Soft Fencing command failed on host battlestar.tlv.redhat.com: null >Stdout: >Stderr: >Stacktrace: java.nio.channels.UnresolvedAddressException: java.nio.channels.UnresolvedAddressException > at sun.nio.ch.Net.checkAddress(Net.java:127) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.implConnect(UnixAsynchronousSocketChannelImpl.java:301) [rt.jar:1.7.0_25] > at sun.nio.ch.AsynchronousSocketChannelImpl.connect(AsynchronousSocketChannelImpl.java:208) [rt.jar:1.7.0_25] > at org.apache.sshd.common.io.nio2.Nio2Connector.connect(Nio2Connector.java:47) > at org.apache.sshd.SshClient.connect(SshClient.java:334) > at org.apache.sshd.SshClient.connect(SshClient.java:320) > at org.apache.sshd.SshClient.connect(SshClient.java:310) > at org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:284) [uutils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:87) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:53) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1170) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1921) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:347) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view8.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$4.run(VdsEventListener.java:240) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:29:39,585 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-7) [16b84ca3] Lock Acquired to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-13 10:29:39,586 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-7) [16b84ca3] Fence failed on vds h1 which is spm of pool 00000002-0002-0002-0002-00000000037f - moving pool to non operational >2014-08-13 10:29:39,596 WARN [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (org.ovirt.thread.pool-8-thread-7) [728d6977] The message key SetStoragePoolStatus is missing from bundles/ExecutionMessages >2014-08-13 10:29:39,646 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-8-thread-7) [728d6977] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 10:29:39,936 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-7) [728d6977] Correlation ID: 728d6977, Job ID: 0d08316e-6776-4bd7-bb7c-9369efdcf338, Call Stack: null, Custom Event ID: -1, Message: Fencing failed on Storage Pool Manager h1 for Data Center Default. Setting status to Non-Operational. >2014-08-13 10:29:39,953 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-8-thread-7) [728d6977] Lock freed to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-13 10:29:39,955 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-7) [728d6977] Failed to run Fence script on vds:h1. >2014-08-13 10:29:39,964 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-7) [728d6977] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-08-13 10:29:39,965 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-7) [728d6977] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-08-13 10:29:39,966 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-7) [728d6977] Trying to release exclusive lock which does not exist, lock key: 9c67d903-8424-4cda-8bb1-3d8652b92d77VDS_FENCE >2014-08-13 10:29:39,967 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-7) [728d6977] Lock freed to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-13 10:29:42,176 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:29:42,178 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:29:42,246 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:29:45,773 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:29:45,773 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:29:45,825 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:29:48,860 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:29:48,862 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:29:48,914 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:29:51,948 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:29:51,949 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:29:52,020 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:29:55,055 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:29:55,056 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:29:55,657 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:29:59,096 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:29:59,097 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:29:59,151 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:30:02,171 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:30:02,172 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:30:02,231 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:30:05,250 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:30:05,251 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:30:05,321 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:30:08,342 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:30:08,343 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:30:08,369 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:30:11,903 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:30:11,905 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:30:11,938 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:30:14,957 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:30:14,959 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:30:15,011 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:30:18,034 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:30:18,035 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:30:18,083 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 10:30:21,102 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-13 10:30:21,103 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host h1 is not responding. It will stay in Connecting state for a grace period of $160 seconds and after that an attempt to fence the host will be issued. >2014-08-13 10:30:21,222 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:628) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.UnknownHostException: battlestar.tlv.redhat.com > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:178) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 11:01:25,611 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-13 11:01:25,640 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-13 11:01:25,641 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-13 11:01:25,642 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-13 11:01:25,643 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-13 11:01:25,644 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-13 11:01:25,645 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-13 11:01:25,645 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-13 11:01:25,646 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-13 11:01:25,646 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-13 11:01:25,647 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-13 11:01:25,648 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-13 11:01:25,648 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-13 11:01:25,649 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-13 11:01:25,649 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-13 11:01:25,650 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-13 11:01:25,650 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-13 11:01:25,651 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-13 11:01:25,651 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-13 11:01:25,652 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-13 11:01:25,652 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-13 11:01:25,653 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-13 11:01:25,653 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DB_USER" is "engine". >2014-08-13 11:01:25,654 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-13 11:01:25,654 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-13 11:01:25,655 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-13 11:01:25,656 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-13 11:01:25,657 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-13 11:01:25,657 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-13 11:01:25,658 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-13 11:01:25,658 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-13 11:01:25,659 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-13 11:01:25,659 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-13 11:01:25,660 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-13 11:01:25,660 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-13 11:01:25,661 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-13 11:01:25,661 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-13 11:01:25,662 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-13 11:01:25,662 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-13 11:01:25,663 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-13 11:01:25,663 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-13 11:01:25,664 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-13 11:01:25,665 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-13 11:01:25,665 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-13 11:01:25,666 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-13 11:01:25,666 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-13 11:01:25,667 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-13 11:01:25,668 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-13 11:01:25,668 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-13 11:01:25,669 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-13 11:01:25,669 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-13 11:01:25,671 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-13 11:01:25,677 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-13 11:01:25,683 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-13 11:01:25,684 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-13 11:01:25,685 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-13 11:01:25,685 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-13 11:01:25,686 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-13 11:01:25,686 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-13 11:01:25,687 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-13 11:01:25,689 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-13 11:01:25,690 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-13 11:01:25,690 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-13 11:01:25,691 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_USER" is "ovirt". >2014-08-13 11:01:25,691 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-13 11:01:25,692 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-13 11:01:25,692 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-13 11:01:25,693 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-13 11:01:25,694 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-13 11:01:25,694 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-8) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-13 11:01:26,765 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-2) Start initializing Backend >2014-08-13 11:01:27,144 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-2) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 11:01:27,281 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-2) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 11:01:27,314 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-2) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-13 11:01:27,443 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-2) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-13 11:01:27,888 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-2) Running ovirt-engine 3.5.0.0 >2014-08-13 11:01:27,888 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-2) Start initializing dictionaries >2014-08-13 11:01:27,891 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-2) Finished initializing dictionaries >2014-08-13 11:01:27,891 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Start initializing AuditLogCleanupManager >2014-08-13 11:01:27,892 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-13 11:01:27,950 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Finished initializing AuditLogCleanupManager >2014-08-13 11:01:27,950 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Start initializing CommandEntityCleanupManager >2014-08-13 11:01:27,951 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-13 11:01:27,952 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Finished initializing CommandEntityCleanupManager >2014-08-13 11:01:27,954 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-2) Start initializing TagsDirector >2014-08-13 11:01:27,964 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-2) Tag root added to tree >2014-08-13 11:01:28,017 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-2) Finished initializing TagsDirector >2014-08-13 11:01:28,018 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-2) Start initializing IsoDomainListSyncronizer >2014-08-13 11:01:28,024 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-2) Finished initializing IsoDomainListSyncronizer >2014-08-13 11:01:28,110 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-2) Completed initializing handlers >2014-08-13 11:01:28,130 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Start initializing ErrorTranslatorImpl >2014-08-13 11:01:28,140 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-13 11:01:28,142 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Finished initializing ErrorTranslatorImpl >2014-08-13 11:01:28,142 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Start initializing ErrorTranslatorImpl >2014-08-13 11:01:28,143 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Finished initializing ErrorTranslatorImpl >2014-08-13 11:01:28,143 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-2) Mark incomplete jobs as UNKNOWN >2014-08-13 11:01:28,305 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-2) Start initializing JobRepositoryCleanupManager >2014-08-13 11:01:28,306 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-2) Finished initializing JobRepositoryCleanupManager >2014-08-13 11:01:28,307 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-2) Start initializing AutoRecoveryManager >2014-08-13 11:01:28,308 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-2) Finished initializing AutoRecoveryManager >2014-08-13 11:01:28,309 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-2) Start initializing ExecutionMessageDirector >2014-08-13 11:01:28,311 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-2) Finished initializing ExecutionMessageDirector >2014-08-13 11:01:28,759 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-2) Start of enabled extensions list >2014-08-13 11:01:28,759 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-2) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-13 11:01:28,760 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-2) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-13 11:01:28,761 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-2) End of enabled extensions list >2014-08-13 11:01:28,942 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-4) Start initializing DbUserCacheManager >2014-08-13 11:01:28,943 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-4) Finished initializing DbUserCacheManager >2014-08-13 11:01:28,979 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-13 11:01:28,987 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-4) Initialization of AsyncTaskManager completed successfully. >2014-08-13 11:01:28,988 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-4) Start initializing ResourceManager >2014-08-13 11:01:29,138 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-13 11:01:29,140 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: UNASSIGNED not exist in string table >2014-08-13 11:01:29,141 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-13 11:01:29,141 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-13 11:01:29,142 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-13 11:01:29,142 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-13 11:01:29,142 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-13 11:01:29,143 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-13 11:01:29,143 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-13 11:01:29,144 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-13 11:01:29,144 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-13 11:01:29,145 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-13 11:01:29,145 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: DWH_STOPPED not exist in string table >2014-08-13 11:01:29,145 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: DWH_STARTED not exist in string table >2014-08-13 11:01:29,145 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: DWH_ERROR not exist in string table >2014-08-13 11:01:29,146 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-13 11:01:29,146 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-13 11:01:29,146 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-13 11:01:29,147 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-13 11:01:29,653 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm3 was set to the Unknown status. >2014-08-13 11:01:29,699 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-4) Entered VdsManager constructor >2014-08-13 11:01:29,726 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-4) Initialize vdsBroker (galactica,54,321) >2014-08-13 11:01:29,767 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-4) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-13 11:01:29,767 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-4) Entered VdsManager constructor >2014-08-13 11:01:29,775 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-4) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-13 11:01:29,922 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-4) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-13 11:01:29,940 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-4) Finished initializing ResourceManager >2014-08-13 11:01:29,944 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-4) Initialization of OvfDataUpdater completed successfully. >2014-08-13 11:01:29,945 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Start scheduling to enable vds load balancer >2014-08-13 11:01:29,945 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Finished scheduling to enable vds load balancer >2014-08-13 11:01:29,946 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Start HA Reservation check >2014-08-13 11:01:29,946 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Finished HA Reservation check >2014-08-13 11:01:29,956 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-4) Init VM custom properties utilities >2014-08-13 11:01:29,957 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-4) Init device custom properties utilities >2014-08-13 11:01:29,960 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Initializing Scheduling manager >2014-08-13 11:01:30,010 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-13 11:01:30,010 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-13 11:01:30,011 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-13 11:01:30,022 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) External scheduler disabled, discovery skipped >2014-08-13 11:01:30,023 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Initialized Scheduling manager >2014-08-13 11:01:30,024 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-4) Initializing DWH Heart Beat >2014-08-13 11:01:30,025 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-4) DWH Heart Beat initialized >2014-08-13 11:01:32,787 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: h2 >2014-08-13 11:01:34,380 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-8) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 452881d7 >2014-08-13 11:01:34,732 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, GetHardwareInfoVDSCommand, log id: 452881d7 >2014-08-13 11:01:34,889 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-13 11:01:34,890 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) Host h1 is running with disabled SELinux. >2014-08-13 11:01:35,289 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-8) [2fddb2aa] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-13 11:01:35,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-8) [2fddb2aa] START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 2cd2cd97 >2014-08-13 11:01:35,960 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-8) [2fddb2aa] FINISH, GetHardwareInfoVDSCommand, log id: 2cd2cd97 >2014-08-13 11:01:35,981 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [2fddb2aa] Host h1 is running with disabled SELinux. >2014-08-13 11:01:36,172 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) [2fddb2aa] VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from Unknown --> Up >2014-08-13 11:01:36,264 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) [2fddb2aa] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm3 status was restored to Up. >2014-08-13 11:01:36,502 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-8) [36801430] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 11:01:36,554 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-8) [1afc8e23] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 11:01:36,709 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-8) [1afc8e23] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = NFS, connectionList = [{ id: 1670ace3-7db6-4c73-9b3c-614dcc85bd46, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc5, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc4, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 189eeeee >2014-08-13 11:01:36,990 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-8) [1afc8e23] FINISH, ConnectStorageServerVDSCommand, return: {1670ace3-7db6-4c73-9b3c-614dcc85bd46=0, 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a=0}, log id: 189eeeee >2014-08-13 11:01:36,992 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-8) [1afc8e23] Host h1 storage connection was succeeded >2014-08-13 11:01:37,016 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-9) START, ConnectStoragePoolVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vdsId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, masterVersion = 1), log id: 2bb78f1d >2014-08-13 11:01:37,802 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-9) FINISH, ConnectStoragePoolVDSCommand, log id: 2bb78f1d >2014-08-13 11:01:38,344 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-8) [1afc8e23] START, SetMOMPolicyParametersVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 2b722939 >2014-08-13 11:01:38,578 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-8) [1afc8e23] FINISH, SetMOMPolicyParametersVDSCommand, log id: 2b722939 >2014-08-13 11:01:38,616 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) [1afc8e23] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host h1 was set to Up. >2014-08-13 11:01:38,720 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-8) [1afc8e23] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h1. >2014-08-13 11:01:38,935 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [1afc8e23] Initializing Host: h1 >2014-08-13 11:01:38,953 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-8) [51afb46] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-13 11:01:40,380 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-22) hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-13 11:01:40,405 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-22) Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-13 11:01:40,438 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-13 11:01:40,458 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-19) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 5cdadff6 >2014-08-13 11:01:41,672 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 42 GB of free space >2014-08-13 11:01:41,705 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 42 GB of free space >2014-08-13 11:01:41,706 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-19) -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-13 11:01:41,708 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-19) START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 1429f8a5 >2014-08-13 11:01:41,999 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-19) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 1429f8a5 >2014-08-13 11:01:42,000 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-19) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 5cdadff6 >2014-08-13 11:01:42,001 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-19) Discovered no tasks on Storage Pool Default >2014-08-13 11:07:40,758 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-1) Running command: LoginAdminUserCommand internal: false. >2014-08-13 11:07:40,796 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-13 11:07:44,566 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-4) Running command: LoginUserCommand internal: false. >2014-08-13 11:08:04,012 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-1) [5d6707d] Lock Acquired to object EngineLock [exclusiveLocks= key: de580b19-e975-426f-80de-6c7daf1f24ab value: VM >, sharedLocks= ] >2014-08-13 11:08:04,162 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-1) [5d6707d] START, IsVmDuringInitiatingVDSCommand( vmId = de580b19-e975-426f-80de-6c7daf1f24ab), log id: 712d6003 >2014-08-13 11:08:04,163 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-1) [5d6707d] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 712d6003 >2014-08-13 11:08:04,416 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-34) [5d6707d] Running command: RunVmCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-13 11:08:04,491 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-34) [5d6707d] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@2fe0f043), log id: 4a93b65c >2014-08-13 11:08:04,499 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-34) [5d6707d] FINISH, UpdateVmDynamicDataVDSCommand, log id: 4a93b65c >2014-08-13 11:08:04,516 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-34) [5d6707d] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, vm=VM [vm4]), log id: 5f466e57 >2014-08-13 11:08:04,602 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-34) [5d6707d] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, vm=VM [vm4]), log id: 52ae8e8f >2014-08-13 11:08:04,662 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-34) [5d6707d] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=de580b19-e975-426f-80de-6c7daf1f24ab,acpiEnable=true,custom={device_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2device_01ed0eff-5cbe-4be1-bbb9-7f79ea757c8adevice_abf905a2-ecae-45d8-a5ed-496f7ec0771adevice_fc97fde1-adde-4569-a959-7a83b47cdc15device_f38973f9-5bf7-4699-85d3-bcb7fa74541c=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=f38973f9-5bf7-4699-85d3-bcb7fa74541c, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2device_01ed0eff-5cbe-4be1-bbb9-7f79ea757c8a=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=01ed0eff-5cbe-4be1-bbb9-7f79ea757c8a, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2device_01ed0eff-5cbe-4be1-bbb9-7f79ea757c8adevice_abf905a2-ecae-45d8-a5ed-496f7ec0771a=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=abf905a2-ecae-45d8-a5ed-496f7ec0771a, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2device_01ed0eff-5cbe-4be1-bbb9-7f79ea757c8adevice_abf905a2-ecae-45d8-a5ed-496f7ec0771adevice_fc97fde1-adde-4569-a959-7a83b47cdc15=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=fc97fde1-adde-4569-a959-7a83b47cdc15, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={ram=65536, vram=32768, heads=1}, device=qxl, type=video, deviceId=b3b178bf-b83c-4659-a950-b4e8762616e5}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=5e9cb4a3-8653-4e26-80c9-ff9a205bacea}, {shared=false, index=0, volumeID=c70c663c-765a-4808-b98d-af453f8cc170, propagateErrors=off, format=cow, type=disk, iface=virtio, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageID=016a7ddb-9007-4925-ae72-202649686c94, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=016a7ddb-9007-4925-ae72-202649686c94}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:54, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=85f4913e-164d-4963-a4f3-c9761543de9e}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=271440b0-d17a-4d84-be46-df27b769e570}, {address={bus=0x00, domain=0x0000, slot=0x05, type=pci, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2}],vmName=vm4,cpuType=SandyBridge,fileTransferEnable=true >2014-08-13 11:08:05,133 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-34) [5d6707d] FINISH, CreateVDSCommand, log id: 52ae8e8f >2014-08-13 11:08:05,186 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-34) [5d6707d] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 5f466e57 >2014-08-13 11:08:05,187 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-34) [5d6707d] Lock freed to object EngineLock [exclusiveLocks= key: de580b19-e975-426f-80de-6c7daf1f24ab value: VM >, sharedLocks= ] >2014-08-13 11:08:05,244 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-34) [5d6707d] Correlation ID: 5d6707d, Job ID: 324a424d-554c-4271-9899-355437c4f838, Call Stack: null, Custom Event ID: -1, Message: VM vm4 was started by admin (Host: h1). >2014-08-13 11:08:08,905 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) VM vm4 de580b19-e975-426f-80de-6c7daf1f24ab moved from WaitForLaunch --> PoweringUp >2014-08-13 11:08:08,906 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-82) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[de580b19-e975-426f-80de-6c7daf1f24ab]), log id: 5eac89d1 >2014-08-13 11:08:09,355 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-82) FINISH, FullListVdsCommand, return: [{displaySecurePort=5901, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.35.0.105, pauseCode=NOERR, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, acpiEnable=true, custom={device_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2device_01ed0eff-5cbe-4be1-bbb9-7f79ea757c8adevice_abf905a2-ecae-45d8-a5ed-496f7ec0771adevice_fc97fde1-adde-4569-a959-7a83b47cdc15device_f38973f9-5bf7-4699-85d3-bcb7fa74541c=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=f38973f9-5bf7-4699-85d3-bcb7fa74541c, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2device_01ed0eff-5cbe-4be1-bbb9-7f79ea757c8a=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=01ed0eff-5cbe-4be1-bbb9-7f79ea757c8a, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2device_01ed0eff-5cbe-4be1-bbb9-7f79ea757c8adevice_abf905a2-ecae-45d8-a5ed-496f7ec0771a=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=abf905a2-ecae-45d8-a5ed-496f7ec0771a, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_e6b1ecdc-a963-4ae6-b9cf-0be2851b33e2device_01ed0eff-5cbe-4be1-bbb9-7f79ea757c8adevice_abf905a2-ecae-45d8-a5ed-496f7ec0771adevice_fc97fde1-adde-4569-a959-7a83b47cdc15=VmDevice {vmId=de580b19-e975-426f-80de-6c7daf1f24ab, deviceId=fc97fde1-adde-4569-a959-7a83b47cdc15, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=28899, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@7c445187, vmName=vm4, fileTransferEnable=true, cpuType=SandyBridge}], log id: 5eac89d1 >2014-08-13 11:08:09,369 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Received a memballoon Device without an address when processing VM de580b19-e975-426f-80de-6c7daf1f24ab devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-08-13 11:08:09,370 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Received a spice Device without an address when processing VM de580b19-e975-426f-80de-6c7daf1f24ab devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5901, type=graphics} >2014-08-13 11:08:16,711 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (http--0.0.0.0-8080-2) [48a6ac16] Running command: SetVmTicketCommand internal: false. Entities affected : ID: de580b19-e975-426f-80de-6c7daf1f24ab Type: VMAction group CONNECT_TO_VM with role type USER >2014-08-13 11:08:16,749 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-2) [48a6ac16] START, SetVmTicketVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=de580b19-e975-426f-80de-6c7daf1f24ab, ticket=I7lo7bR9oAGz, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 2dc3fec7 >2014-08-13 11:08:17,085 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-2) [48a6ac16] FINISH, SetVmTicketVDSCommand, log id: 2dc3fec7 >2014-08-13 11:08:17,180 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [48a6ac16] Correlation ID: 48a6ac16, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM vm4 >2014-08-13 11:08:24,095 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-24) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User <UNKNOWN> is connected to VM vm4. >2014-08-13 11:09:06,274 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) VM vm4 de580b19-e975-426f-80de-6c7daf1f24ab moved from PoweringUp --> Up >2014-08-13 11:09:06,395 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-36) Correlation ID: 5d6707d, Job ID: 324a424d-554c-4271-9899-355437c4f838, Call Stack: null, Custom Event ID: -1, Message: VM vm4 started on Host h1 >2014-08-13 11:09:47,819 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (http--0.0.0.0-8080-1) [5d027266] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group CONNECT_TO_VM with role type USER >2014-08-13 11:09:47,828 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-1) [5d027266] START, SetVmTicketVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, ticket=OgO192A9fzSa, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 62382a9d >2014-08-13 11:09:48,098 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-1) [5d027266] FINISH, SetVmTicketVDSCommand, log id: 62382a9d >2014-08-13 11:09:48,160 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [5d027266] Correlation ID: 5d027266, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM vm3 >2014-08-13 11:09:56,608 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User <UNKNOWN> got disconnected from VM vm4. >2014-08-13 11:10:13,233 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-31) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin is connected to VM vm3. >2014-08-13 11:10:29,633 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-8-thread-43) [24bc236c] Running command: StopVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-13 11:10:29,637 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-43) [24bc236c] START, DestroyVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, force=false, secondsToWait=0, gracefully=false, reason=), log id: 1c0bf980 >2014-08-13 11:10:30,006 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-68) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin got disconnected from VM vm3. >2014-08-13 11:10:30,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-43) [24bc236c] START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, force=false, secondsToWait=0, gracefully=false, reason=), log id: 7a754bd1 >2014-08-13 11:10:30,561 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-8-thread-43) [24bc236c] FINISH, DestroyVDSCommand, log id: 7a754bd1 >2014-08-13 11:10:30,660 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-8-thread-43) [24bc236c] FINISH, DestroyVmVDSCommand, return: Down, log id: 1c0bf980 >2014-08-13 11:10:30,690 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-45) [6a311584] Running command: ProcessDownVmCommand internal: true. >2014-08-13 11:10:30,755 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-43) [24bc236c] Correlation ID: 24bc236c, Job ID: cb44d85a-efbb-4db9-a6ed-af63169b0c41, Call Stack: null, Custom Event ID: -1, Message: VM vm3 powered off by admin (Host: h1) (Reason: Not Specified). >2014-08-13 11:10:55,251 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (http--0.0.0.0-8080-1) [34cb42d5] Lock Acquired to object EngineLock [exclusiveLocks= key: vm3 value: VM_NAME >, sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-13 11:10:55,317 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (http--0.0.0.0-8080-1) [34cb42d5] Running command: UpdateVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group EDIT_VM_PROPERTIES with role type USER >2014-08-13 11:10:55,409 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [34cb42d5] Correlation ID: 34cb42d5, Job ID: eca665d6-e263-4790-9d2a-f810d26fe5a8, Call Stack: null, Custom Event ID: -1, Message: VM vm3 configuration was updated by admin. >2014-08-13 11:10:55,417 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (http--0.0.0.0-8080-1) [34cb42d5] Lock freed to object EngineLock [exclusiveLocks= key: vm3 value: VM_NAME >, sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-13 11:11:04,485 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-7) [73db2e1d] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-13 11:11:04,523 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-7) [73db2e1d] START, IsVmDuringInitiatingVDSCommand( vmId = 5b1ee280-4251-489a-899c-8e9dac73c327), log id: 51f07372 >2014-08-13 11:11:04,525 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-7) [73db2e1d] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 51f07372 >2014-08-13 11:11:04,611 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-24) [73db2e1d] Running command: RunVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-13 11:11:04,694 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-24) [73db2e1d] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = ISCSI, connectionList = [{ id: bd327084-78e4-4b84-9a74-9cea2d43c971, connection: 10.35.16.25, iqn: iqn.2011-05.com.redhat:achub-iscsi, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 26fb80a7 >2014-08-13 11:11:05,496 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-24) [73db2e1d] FINISH, ConnectStorageServerVDSCommand, return: {bd327084-78e4-4b84-9a74-9cea2d43c971=0}, log id: 26fb80a7 >2014-08-13 11:11:05,504 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-24) [73db2e1d] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@9ea5b648), log id: 39b9660b >2014-08-13 11:11:05,662 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-24) [73db2e1d] FINISH, UpdateVmDynamicDataVDSCommand, log id: 39b9660b >2014-08-13 11:11:05,666 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-24) [73db2e1d] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, vm=VM [vm3]), log id: 4985ac42 >2014-08-13 11:11:05,683 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-24) [73db2e1d] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, vm=VM [vm3]), log id: e653bd >2014-08-13 11:11:05,701 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-24) [73db2e1d] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=5b1ee280-4251-489a-899c-8e9dac73c327,acpiEnable=true,custom={device_a0176251-79b2-48df-ba52-49dc7fd73498=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=a0176251-79b2-48df-ba52-49dc7fd73498, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48device_611f9766-6112-4e87-86c1-42cc5f30a38c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=611f9766-6112-4e87-86c1-42cc5f30a38c, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48device_611f9766-6112-4e87-86c1-42cc5f30a38cdevice_96c8ccf7-eae6-4354-ac32-442f636eee8c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=2d383e02-c4b2-4b55-b54c-60d73acbce48, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=f09597d8-8cc8-4549-863f-b96975014b01}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=f76be01f-4e52-4d67-8e1b-0856337e9929}, {shared=false, index=0, volumeID=26308abd-8540-410e-8018-58cd2157084d, propagateErrors=off, format=cow, type=disk, iface=virtio, bootOrder=1, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, imageID=b5576e45-7f13-465f-b26f-05d52faf28d1, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=b5576e45-7f13-465f-b26f-05d52faf28d1}, {shared=false, iface=virtio, GUID=3600144f09dbd050000004e1994c60005, address={bus=0x00, domain=0x0000, slot=0x07, type=pci, function=0x0}, specParams={}, optional=false, propagateErrors=off, device=lun, format=raw, type=disk, readonly=false, deviceId=bdca3776-6597-4323-84f1-4ea75f7d269d}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:53, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=68e93502-4ad4-42a0-9a23-dfa3f67090f5}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=403887de-d0b8-4108-8b43-b4b5d5413859}, {address={bus=0x00, domain=0x0000, slot=0x05, type=pci, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c}],vmName=vm3,cpuType=SandyBridge,fileTransferEnable=true >2014-08-13 11:11:06,167 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-24) [73db2e1d] FINISH, CreateVDSCommand, log id: e653bd >2014-08-13 11:11:06,292 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-24) [73db2e1d] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 4985ac42 >2014-08-13 11:11:06,292 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-24) [73db2e1d] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-13 11:11:06,316 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-24) [73db2e1d] Correlation ID: 73db2e1d, Job ID: 6e096a3f-0d89-42ba-aa0c-3e6b6d516918, Call Stack: null, Custom Event ID: -1, Message: VM vm3 was started by admin (Host: h1). >2014-08-13 11:11:09,040 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from WaitForLaunch --> PoweringUp >2014-08-13 11:11:09,041 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-74) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: 23b2fa71 >2014-08-13 11:11:09,346 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-74) FINISH, FullListVdsCommand, return: [{displaySecurePort=5900, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.35.0.105, pauseCode=NOERR, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, acpiEnable=true, custom={device_a0176251-79b2-48df-ba52-49dc7fd73498=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=a0176251-79b2-48df-ba52-49dc7fd73498, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48device_611f9766-6112-4e87-86c1-42cc5f30a38c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=611f9766-6112-4e87-86c1-42cc5f30a38c, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48device_611f9766-6112-4e87-86c1-42cc5f30a38cdevice_96c8ccf7-eae6-4354-ac32-442f636eee8c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=2d383e02-c4b2-4b55-b54c-60d73acbce48, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=29311, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@3390570f, vmName=vm3, fileTransferEnable=true, cpuType=SandyBridge}], log id: 23b2fa71 >2014-08-13 11:11:09,368 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Received a memballoon Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-08-13 11:11:09,370 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Received a spice Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5900, type=graphics} >2014-08-13 11:11:14,945 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (http--0.0.0.0-8080-1) [184c693b] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group CONNECT_TO_VM with role type USER >2014-08-13 11:11:14,950 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-1) [184c693b] START, SetVmTicketVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, ticket=/wD72nutt3Gz, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 45386cc9 >2014-08-13 11:11:15,168 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (http--0.0.0.0-8080-1) [184c693b] FINISH, SetVmTicketVDSCommand, log id: 45386cc9 >2014-08-13 11:11:15,237 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [184c693b] Correlation ID: 184c693b, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM vm3 >2014-08-13 11:11:25,209 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin is connected to VM vm3. >2014-08-13 11:12:09,582 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from PoweringUp --> Up >2014-08-13 11:12:09,715 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) Correlation ID: 73db2e1d, Job ID: 6e096a3f-0d89-42ba-aa0c-3e6b6d516918, Call Stack: null, Custom Event ID: -1, Message: VM vm3 started on Host h1 >2014-08-13 11:55:50,676 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-13 11:55:50,717 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-13 11:55:50,718 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-13 11:55:50,719 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-13 11:55:50,720 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-13 11:55:50,721 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-13 11:55:50,722 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-13 11:55:50,723 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-13 11:55:50,723 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-13 11:55:50,724 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-13 11:55:50,724 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-13 11:55:50,725 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-13 11:55:50,725 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-13 11:55:50,726 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-13 11:55:50,726 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-13 11:55:50,727 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-13 11:55:50,727 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-13 11:55:50,728 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-13 11:55:50,728 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-13 11:55:50,729 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-13 11:55:50,729 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-13 11:55:50,730 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-13 11:55:50,730 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_USER" is "engine". >2014-08-13 11:55:50,731 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-13 11:55:50,731 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-13 11:55:50,732 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-13 11:55:50,732 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-13 11:55:50,733 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-13 11:55:50,734 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-13 11:55:50,734 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-13 11:55:50,735 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-13 11:55:50,735 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-13 11:55:50,735 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-13 11:55:50,736 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-13 11:55:50,736 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-13 11:55:50,737 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-13 11:55:50,737 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-13 11:55:50,738 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-13 11:55:50,739 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-13 11:55:50,739 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-13 11:55:50,740 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-13 11:55:50,740 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-13 11:55:50,741 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-13 11:55:50,741 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-13 11:55:50,742 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-13 11:55:50,742 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-13 11:55:50,743 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-13 11:55:50,743 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-13 11:55:50,744 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-13 11:55:50,744 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-13 11:55:50,745 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-13 11:55:50,746 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-13 11:55:50,747 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-13 11:55:50,747 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-13 11:55:50,747 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-13 11:55:50,748 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-13 11:55:50,748 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-13 11:55:50,748 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-13 11:55:50,749 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-13 11:55:50,750 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-13 11:55:50,750 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-13 11:55:50,750 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-13 11:55:50,751 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-13 11:55:50,751 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_USER" is "ovirt". >2014-08-13 11:55:50,752 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-13 11:55:50,752 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-13 11:55:50,753 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-13 11:55:50,753 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-13 11:55:50,754 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-13 11:55:50,754 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-13 11:55:51,575 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-8) Start initializing Backend >2014-08-13 11:55:51,878 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-8) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 11:55:51,978 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-8) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 11:55:52,021 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-8) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-13 11:55:52,162 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-8) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-13 11:55:52,575 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-8) Running ovirt-engine 3.5.0.0 >2014-08-13 11:55:52,576 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-8) Start initializing dictionaries >2014-08-13 11:55:52,580 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-8) Finished initializing dictionaries >2014-08-13 11:55:52,581 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-8) Start initializing AuditLogCleanupManager >2014-08-13 11:55:52,582 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-8) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-13 11:55:52,636 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-8) Finished initializing AuditLogCleanupManager >2014-08-13 11:55:52,636 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-8) Start initializing CommandEntityCleanupManager >2014-08-13 11:55:52,637 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-8) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-13 11:55:52,638 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-8) Finished initializing CommandEntityCleanupManager >2014-08-13 11:55:52,639 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-8) Start initializing TagsDirector >2014-08-13 11:55:52,646 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-8) Tag root added to tree >2014-08-13 11:55:52,679 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-8) Finished initializing TagsDirector >2014-08-13 11:55:52,680 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-8) Start initializing IsoDomainListSyncronizer >2014-08-13 11:55:52,685 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-8) Finished initializing IsoDomainListSyncronizer >2014-08-13 11:55:52,762 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-8) Completed initializing handlers >2014-08-13 11:55:52,781 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-8) Start initializing ErrorTranslatorImpl >2014-08-13 11:55:52,792 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-8) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-13 11:55:52,794 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-8) Finished initializing ErrorTranslatorImpl >2014-08-13 11:55:52,794 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-8) Start initializing ErrorTranslatorImpl >2014-08-13 11:55:52,794 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-8) Finished initializing ErrorTranslatorImpl >2014-08-13 11:55:52,795 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-8) Mark incomplete jobs as UNKNOWN >2014-08-13 11:55:52,881 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-8) Start initializing JobRepositoryCleanupManager >2014-08-13 11:55:52,882 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-8) Finished initializing JobRepositoryCleanupManager >2014-08-13 11:55:52,882 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-8) Start initializing AutoRecoveryManager >2014-08-13 11:55:52,883 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-8) Finished initializing AutoRecoveryManager >2014-08-13 11:55:52,885 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-8) Start initializing ExecutionMessageDirector >2014-08-13 11:55:52,886 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-8) Finished initializing ExecutionMessageDirector >2014-08-13 11:55:53,214 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-8) Start of enabled extensions list >2014-08-13 11:55:53,214 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-8) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-13 11:55:53,215 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-8) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-13 11:55:53,216 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-8) End of enabled extensions list >2014-08-13 11:55:53,346 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-8) Start initializing DbUserCacheManager >2014-08-13 11:55:53,347 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-8) Finished initializing DbUserCacheManager >2014-08-13 11:55:53,381 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-13 11:55:53,384 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-8) Initialization of AsyncTaskManager completed successfully. >2014-08-13 11:55:53,385 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-8) Start initializing ResourceManager >2014-08-13 11:55:53,491 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-8) Entered VdsManager constructor >2014-08-13 11:55:53,531 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-8) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-13 11:55:53,707 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-8) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-13 11:55:53,708 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-8) Entered VdsManager constructor >2014-08-13 11:55:53,717 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-8) Initialize vdsBroker (galactica,54,321) >2014-08-13 11:55:53,737 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-8) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-13 11:55:53,753 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-8) Finished initializing ResourceManager >2014-08-13 11:55:53,757 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-8) Initialization of OvfDataUpdater completed successfully. >2014-08-13 11:55:53,757 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) Start scheduling to enable vds load balancer >2014-08-13 11:55:53,758 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) Finished scheduling to enable vds load balancer >2014-08-13 11:55:53,758 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) Start HA Reservation check >2014-08-13 11:55:53,758 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) Finished HA Reservation check >2014-08-13 11:55:53,764 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-8) Init VM custom properties utilities >2014-08-13 11:55:53,764 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-8) Init device custom properties utilities >2014-08-13 11:55:53,768 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) Initializing Scheduling manager >2014-08-13 11:55:53,824 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) External scheduler disabled, discovery skipped >2014-08-13 11:55:53,824 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-8) Initialized Scheduling manager >2014-08-13 11:55:53,825 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-8) Initializing DWH Heart Beat >2014-08-13 11:55:53,826 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-8) DWH Heart Beat initialized >2014-08-13 11:55:53,828 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-13 11:55:53,829 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-13 11:55:53,829 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-13 11:55:56,763 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: h2 >2014-08-13 11:55:59,680 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: a58f869 >2014-08-13 11:55:59,981 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: a58f869 >2014-08-13 11:55:59,991 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-13 11:55:59,997 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: UNASSIGNED not exist in string table >2014-08-13 11:55:59,999 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-13 11:56:00,000 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-13 11:56:00,001 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-13 11:56:00,002 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-13 11:56:00,003 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-13 11:56:00,004 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-13 11:56:00,004 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-13 11:56:00,005 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-13 11:56:00,006 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-13 11:56:00,006 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-13 11:56:00,006 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STOPPED not exist in string table >2014-08-13 11:56:00,007 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STARTED not exist in string table >2014-08-13 11:56:00,007 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_ERROR not exist in string table >2014-08-13 11:56:00,007 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-13 11:56:00,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-13 11:56:00,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-13 11:56:00,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-13 11:56:00,540 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-13 11:56:00,541 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is running with disabled SELinux. >2014-08-13 11:56:01,453 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h1 >2014-08-13 11:56:01,635 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [69594bd8] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-13 11:56:01,797 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [69594bd8] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin got disconnected from VM vm3. >2014-08-13 11:56:03,792 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-21) [466af27] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 11:56:03,799 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-21) [466af27] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-13 11:56:03,804 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-21) [466af27] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-13 11:56:03,967 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [466af27] Correlation ID: 466af27, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-13 11:56:04,528 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-21) [466af27] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-13 11:56:04,639 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-21) [466af27] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-13 11:56:04,696 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [466af27] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-13 11:56:04,717 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-11) [466af27] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 430aeee5 >2014-08-13 11:56:06,502 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [466af27] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 41 GB of free space >2014-08-13 11:56:06,535 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [466af27] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 41 GB of free space >2014-08-13 11:56:06,536 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-11) [466af27] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-13 11:56:06,543 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-11) [466af27] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 60ba409c >2014-08-13 11:56:06,834 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-11) [466af27] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 60ba409c >2014-08-13 11:56:06,836 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-11) [466af27] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 430aeee5 >2014-08-13 11:56:06,837 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-11) [466af27] Discovered no tasks on Storage Pool Default >2014-08-13 11:59:03,019 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-2) Running command: LoginAdminUserCommand internal: false. >2014-08-13 11:59:03,048 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-13 11:59:07,225 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-6) Running command: LoginUserCommand internal: false. >2014-08-13 12:04:16,662 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (http--0.0.0.0-8080-3) [e992e85] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >key: clone_vm3 value: VM_NAME >, sharedLocks= ] >2014-08-13 12:04:17,114 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (http--0.0.0.0-8080-3) [e992e85] Running command: AddVmFromSnapshotCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-00000000039f Type: VdsGroupsAction group CREATE_VM with role type USER, ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: StorageAction group CREATE_DISK with role type USER, ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group CREATE_VM with role type USER >2014-08-13 12:04:17,493 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (http--0.0.0.0-8080-3) [e992e85] Locking VM(id = 71cbc0ac-9afd-4692-9237-c73f3629f08c) with compensation. >2014-08-13 12:04:17,496 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (http--0.0.0.0-8080-3) [e992e85] START, SetVmStatusVDSCommand( vmId = 71cbc0ac-9afd-4692-9237-c73f3629f08c, status = ImageLocked, exit status = Normal), log id: 71ef75b5 >2014-08-13 12:04:17,501 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (http--0.0.0.0-8080-3) [e992e85] FINISH, SetVmStatusVDSCommand, log id: 71ef75b5 >2014-08-13 12:04:17,520 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (http--0.0.0.0-8080-3) [e992e85] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >key: clone_vm3 value: VM_NAME >, sharedLocks= ] >2014-08-13 12:04:17,538 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (http--0.0.0.0-8080-3) [72165d44] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-13 12:04:17,761 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (http--0.0.0.0-8080-3) [72165d44] START, CopyImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageId = 3812059c-f566-4975-955f-9937a0885b61, dstImageGroupId = 4b0c7abe-d9b3-4b7f-b5ec-407fa9a7eb0e, vmId = 71cbc0ac-9afd-4692-9237-c73f3629f08c, dstImageId = 5291fb70-5a68-4956-bbb0-fcacb62161f4, imageDescription = , dstStorageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, copyVolumeType = LeafVol, volumeFormat = RAW, preallocate = Sparse, postZero = false, force = false), log id: 47f4ae88 >2014-08-13 12:04:17,763 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (http--0.0.0.0-8080-3) [72165d44] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID >2014-08-13 12:04:17,764 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (http--0.0.0.0-8080-3) [72165d44] -- copyImage parameters: > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > spUUID=00000002-0002-0002-0002-00000000037f > vmGUID=71cbc0ac-9afd-4692-9237-c73f3629f08c > srcImageGUID=b5576e45-7f13-465f-b26f-05d52faf28d1 > srcVolUUID=3812059c-f566-4975-955f-9937a0885b61 > dstImageGUID=4b0c7abe-d9b3-4b7f-b5ec-407fa9a7eb0e > dstVolUUID=5291fb70-5a68-4956-bbb0-fcacb62161f4 > descr= > dstSdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f >2014-08-13 12:04:19,361 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (http--0.0.0.0-8080-3) [72165d44] FINISH, CopyImageVDSCommand, return: 1437ecc7-1072-4450-b169-a85254a87d8d, log id: 47f4ae88 >2014-08-13 12:04:19,551 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (http--0.0.0.0-8080-3) [72165d44] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 7c866487-39b1-4c6a-a683-42cd64762d23 >2014-08-13 12:04:19,551 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (http--0.0.0.0-8080-3) [72165d44] CommandMultiAsyncTasks::AttachTask: Attaching task 1437ecc7-1072-4450-b169-a85254a87d8d to command 7c866487-39b1-4c6a-a683-42cd64762d23. >2014-08-13 12:04:19,616 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (http--0.0.0.0-8080-3) [72165d44] Adding task 1437ecc7-1072-4450-b169-a85254a87d8d (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 12:04:20,156 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-3) [72165d44] Correlation ID: e992e85, Job ID: 64647734-10f8-4d64-be03-fe058fa1a257, Call Stack: null, Custom Event ID: -1, Message: VM clone_vm3 creation was initiated by admin. >2014-08-13 12:04:20,157 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (http--0.0.0.0-8080-3) [72165d44] BaseAsyncTask::startPollingTask: Starting to poll task 1437ecc7-1072-4450-b169-a85254a87d8d. >2014-08-13 12:04:23,355 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-76) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-13 12:04:23,655 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-76) SPMAsyncTask::PollTask: Polling task 1437ecc7-1072-4450-b169-a85254a87d8d (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-13 12:04:23,657 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-76) Finished polling Tasks, will poll again in 10 seconds. >2014-08-13 12:05:44,593 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-5) [11dd7035] Lock Acquired to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-13 12:05:44,619 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-8) Command d28cff6a-e956-4c74-8fc0-d3ceb3732e04 persisting async task placeholder for child command 35de21a8-2098-48c6-80e9-796eb2aaca0f >2014-08-13 12:05:44,691 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-8) Running command: CreateAllSnapshotsFromVmCommand internal: false. Entities affected : ID: 337241af-a081-4971-a3d2-4c7a790e030d Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER >2014-08-13 12:05:44,749 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-8) [6cbd2cfd] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-08-13 12:05:44,756 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-8) [6cbd2cfd] START, CreateSnapshotVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 2b033ab4-8cd6-4214-baf6-1252264f371e, imageSizeInBytes = 10737418240, volumeFormat = COW, newImageId = dcdd4d3f-02ff-4494-8987-5a5f633927d1, newImageDescription = , imageId = 4024f2e3-f164-402a-b49d-9305398a0aeb, sourceImageGroupId = 2b033ab4-8cd6-4214-baf6-1252264f371e), log id: 3a7f98ba >2014-08-13 12:05:44,760 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-8) [6cbd2cfd] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-13 12:06:00,175 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-8) [6cbd2cfd] FINISH, CreateSnapshotVDSCommand, return: dcdd4d3f-02ff-4494-8987-5a5f633927d1, log id: 3a7f98ba >2014-08-13 12:06:00,219 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-8) [6cbd2cfd] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command d28cff6a-e956-4c74-8fc0-d3ceb3732e04 >2014-08-13 12:06:00,220 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-8) [6cbd2cfd] CommandMultiAsyncTasks::AttachTask: Attaching task 67b91745-4c3f-477d-9fd7-9c289a9bc332 to command d28cff6a-e956-4c74-8fc0-d3ceb3732e04. >2014-08-13 12:06:01,167 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-8) [6cbd2cfd] Adding task 67b91745-4c3f-477d-9fd7-9c289a9bc332 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 12:06:01,473 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-8) Correlation ID: 11dd7035, Job ID: ea577252-57ca-41ee-8a04-f4a9c3e3f9fc, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'snap1' creation for VM 'vm2' was initiated by admin. >2014-08-13 12:06:01,474 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-8) BaseAsyncTask::startPollingTask: Starting to poll task 67b91745-4c3f-477d-9fd7-9c289a9bc332. >2014-08-13 12:06:11,168 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-84) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-13 12:06:11,477 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-84) SPMAsyncTask::PollTask: Polling task 1437ecc7-1072-4450-b169-a85254a87d8d (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 12:06:11,504 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-84) BaseAsyncTask::onTaskEndSuccess: Task 1437ecc7-1072-4450-b169-a85254a87d8d (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 12:06:11,504 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-84) CommandAsyncTask::endActionIfNecessary: All tasks of command 7c866487-39b1-4c6a-a683-42cd64762d23 has ended -> executing endAction >2014-08-13 12:06:11,505 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-84) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 7c866487-39b1-4c6a-a683-42cd64762d23): calling endAction . >2014-08-13 12:06:11,506 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-84) SPMAsyncTask::PollTask: Polling task 67b91745-4c3f-477d-9fd7-9c289a9bc332 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-13 12:06:11,507 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVmFromSnapshot, executionIndex: 0 >2014-08-13 12:06:11,507 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-84) Finished polling Tasks, will poll again in 10 seconds. >2014-08-13 12:06:11,537 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (org.ovirt.thread.pool-8-thread-32) [e992e85] Ending command successfully: org.ovirt.engine.core.bll.AddVmFromSnapshotCommand >2014-08-13 12:06:11,553 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-32) [72165d44] Ending command successfully: org.ovirt.engine.core.bll.CopyImageGroupCommand >2014-08-13 12:06:11,560 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-32) [72165d44] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 4b0c7abe-d9b3-4b7f-b5ec-407fa9a7eb0e, imageId = 5291fb70-5a68-4956-bbb0-fcacb62161f4), log id: 7347eb98 >2014-08-13 12:06:11,860 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-32) [72165d44] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@b38be3b3, log id: 7347eb98 >2014-08-13 12:06:11,868 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-8-thread-32) START, SetVmStatusVDSCommand( vmId = 71cbc0ac-9afd-4692-9237-c73f3629f08c, status = Down, exit status = Normal), log id: 4845908f >2014-08-13 12:06:11,872 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-8-thread-32) FINISH, SetVmStatusVDSCommand, log id: 4845908f >2014-08-13 12:06:11,918 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-32) Correlation ID: e992e85, Call Stack: null, Custom Event ID: -1, Message: VM clone_vm3 creation has been completed. >2014-08-13 12:06:11,918 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVmFromSnapshot completed, handling the result. >2014-08-13 12:06:11,918 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVmFromSnapshot succeeded, clearing tasks. >2014-08-13 12:06:11,934 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-32) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 1437ecc7-1072-4450-b169-a85254a87d8d >2014-08-13 12:06:11,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-32) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 1437ecc7-1072-4450-b169-a85254a87d8d), log id: 404813a5 >2014-08-13 12:06:11,938 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-32) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=1437ecc7-1072-4450-b169-a85254a87d8d), log id: 6327c62f >2014-08-13 12:06:12,418 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-32) FINISH, HSMClearTaskVDSCommand, log id: 6327c62f >2014-08-13 12:06:12,418 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-32) FINISH, SPMClearTaskVDSCommand, log id: 404813a5 >2014-08-13 12:06:12,481 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-32) BaseAsyncTask::removeTaskFromDB: Removed task 1437ecc7-1072-4450-b169-a85254a87d8d from DataBase >2014-08-13 12:06:12,481 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 7c866487-39b1-4c6a-a683-42cd64762d23 >2014-08-13 12:06:21,793 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) SPMAsyncTask::PollTask: Polling task 67b91745-4c3f-477d-9fd7-9c289a9bc332 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 12:06:21,816 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-2) BaseAsyncTask::onTaskEndSuccess: Task 67b91745-4c3f-477d-9fd7-9c289a9bc332 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 12:06:21,817 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-2) CommandAsyncTask::endActionIfNecessary: All tasks of command d28cff6a-e956-4c74-8fc0-d3ceb3732e04 has ended -> executing endAction >2014-08-13 12:06:21,818 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-2) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: d28cff6a-e956-4c74-8fc0-d3ceb3732e04): calling endAction . >2014-08-13 12:06:21,818 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-47) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction CreateAllSnapshotsFromVm, executionIndex: 0 >2014-08-13 12:06:21,857 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-47) Ending command successfully: org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand >2014-08-13 12:06:21,874 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-47) [6cbd2cfd] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotCommand >2014-08-13 12:06:21,883 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-47) [6cbd2cfd] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 2b033ab4-8cd6-4214-baf6-1252264f371e, imageId = dcdd4d3f-02ff-4494-8987-5a5f633927d1), log id: 4a4ca2 >2014-08-13 12:06:22,169 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-47) [6cbd2cfd] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@23e8edc6, log id: 4a4ca2 >2014-08-13 12:06:22,193 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-47) Lock freed to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-13 12:06:22,222 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-47) Correlation ID: 11dd7035, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'snap1' creation for VM 'vm2' has been completed. >2014-08-13 12:06:22,222 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-47) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type CreateAllSnapshotsFromVm completed, handling the result. >2014-08-13 12:06:22,223 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-47) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type CreateAllSnapshotsFromVm succeeded, clearing tasks. >2014-08-13 12:06:22,230 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-47) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 67b91745-4c3f-477d-9fd7-9c289a9bc332 >2014-08-13 12:06:22,231 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-47) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 67b91745-4c3f-477d-9fd7-9c289a9bc332), log id: 76eeeb56 >2014-08-13 12:06:22,232 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-47) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=67b91745-4c3f-477d-9fd7-9c289a9bc332), log id: 41c5f396 >2014-08-13 12:06:22,646 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-47) FINISH, HSMClearTaskVDSCommand, log id: 41c5f396 >2014-08-13 12:06:22,646 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-47) FINISH, SPMClearTaskVDSCommand, log id: 76eeeb56 >2014-08-13 12:06:22,785 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-47) BaseAsyncTask::removeTaskFromDB: Removed task 67b91745-4c3f-477d-9fd7-9c289a9bc332 from DataBase >2014-08-13 12:06:22,785 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-47) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity d28cff6a-e956-4c74-8fc0-d3ceb3732e04 >2014-08-13 12:07:31,168 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-85) Setting new tasks map. The map contains now 0 tasks >2014-08-13 12:07:31,169 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-85) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-13 12:11:03,301 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (http--0.0.0.0-8080-7) [30619db6] Lock Acquired to object EngineLock [exclusiveLocks= key: ggg value: VM_NAME >key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-13 12:17:21,433 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-100) Command SpmStatusVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f) execution failed. Exception: VDSNetworkException: org.apache.commons.httpclient.NoHttpResponseException: The server battlestar.tlv.redhat.com failed to respond >2014-08-13 12:17:21,453 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-100) [69a1e809] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 12:17:21,739 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-100) [69a1e809] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-13 12:17:29,396 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:40) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:35) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:220) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: The connection attempt failed. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:152) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.io.EOFException > at org.postgresql.core.PGStream.ReceiveChar(PGStream.java:276) > at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:271) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108) > ... 39 more > >2014-08-13 12:17:29,415 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-100) [69a1e809] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-13 12:17:29,579 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (http--0.0.0.0-8080-7) [30619db6] Running command: AddVmFromSnapshotCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-00000000039f Type: VdsGroupsAction group CREATE_VM with role type USER, ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: StorageAction group CREATE_DISK with role type USER, ID: 337241af-a081-4971-a3d2-4c7a790e030d Type: VMAction group CREATE_VM with role type USER >2014-08-13 12:17:29,775 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-100) [69a1e809] Correlation ID: 69a1e809, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host h1, Error: Network error during communication with the Host.). >2014-08-13 12:18:35,575 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (http--0.0.0.0-8080-7) [30619db6] Locking VM(id = c7e040ec-6614-4600-9154-594127fdf837) with compensation. >2014-08-13 12:18:35,576 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (http--0.0.0.0-8080-7) [30619db6] START, SetVmStatusVDSCommand( vmId = c7e040ec-6614-4600-9154-594127fdf837, status = ImageLocked, exit status = Normal), log id: 43f2fd94 >2014-08-13 12:18:35,582 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (http--0.0.0.0-8080-7) [30619db6] FINISH, SetVmStatusVDSCommand, log id: 43f2fd94 >2014-08-13 12:18:35,598 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (http--0.0.0.0-8080-7) [30619db6] Lock freed to object EngineLock [exclusiveLocks= key: ggg value: VM_NAME >key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-13 12:18:35,626 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (http--0.0.0.0-8080-7) [13066325] Running command: CopyImageGroupCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-13 12:18:41,555 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (http--0.0.0.0-8080-7) [13066325] START, CopyImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 2b033ab4-8cd6-4214-baf6-1252264f371e, imageId = 4024f2e3-f164-402a-b49d-9305398a0aeb, dstImageGroupId = 2d973705-7786-428d-ab98-25201dbcd8c7, vmId = c7e040ec-6614-4600-9154-594127fdf837, dstImageId = 0c77e8c1-8f7a-4754-be5d-7489a96c704e, imageDescription = , dstStorageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, copyVolumeType = LeafVol, volumeFormat = RAW, preallocate = Sparse, postZero = false, force = false), log id: 33b908dc >2014-08-13 12:18:41,561 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (http--0.0.0.0-8080-7) [13066325] -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID >2014-08-13 12:18:41,562 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (http--0.0.0.0-8080-7) [13066325] -- copyImage parameters: > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > spUUID=00000002-0002-0002-0002-00000000037f > vmGUID=c7e040ec-6614-4600-9154-594127fdf837 > srcImageGUID=2b033ab4-8cd6-4214-baf6-1252264f371e > srcVolUUID=4024f2e3-f164-402a-b49d-9305398a0aeb > dstImageGUID=2d973705-7786-428d-ab98-25201dbcd8c7 > dstVolUUID=0c77e8c1-8f7a-4754-be5d-7489a96c704e > descr= > dstSdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a >2014-08-13 12:18:42,407 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (http--0.0.0.0-8080-7) [13066325] FINISH, CopyImageVDSCommand, return: b9c76de6-a89c-47d2-9e54-a4b936417438, log id: 33b908dc >2014-08-13 12:18:42,439 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (http--0.0.0.0-8080-7) [13066325] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 84464c96-fedf-477b-900e-7c61c7aa1a47 >2014-08-13 12:18:42,440 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (http--0.0.0.0-8080-7) [13066325] CommandMultiAsyncTasks::AttachTask: Attaching task b9c76de6-a89c-47d2-9e54-a4b936417438 to command 84464c96-fedf-477b-900e-7c61c7aa1a47. >2014-08-13 12:18:45,604 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (http--0.0.0.0-8080-7) [13066325] Adding task b9c76de6-a89c-47d2-9e54-a4b936417438 (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 12:18:46,168 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-7) [13066325] Correlation ID: 30619db6, Job ID: 02cec495-2af7-46d8-a572-f030cc474431, Call Stack: null, Custom Event ID: -1, Message: VM ggg creation was initiated by admin. >2014-08-13 12:18:46,168 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (http--0.0.0.0-8080-7) [13066325] BaseAsyncTask::startPollingTask: Starting to poll task b9c76de6-a89c-47d2-9e54-a4b936417438. >2014-08-13 12:18:55,562 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-54) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-13 12:18:55,755 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-54) SPMAsyncTask::PollTask: Polling task b9c76de6-a89c-47d2-9e54-a4b936417438 (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-13 12:18:55,756 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-54) Finished polling Tasks, will poll again in 10 seconds. >2014-08-13 12:29:56,958 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-84) SPMAsyncTask::PollTask: Polling task b9c76de6-a89c-47d2-9e54-a4b936417438 (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 12:29:56,977 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-84) BaseAsyncTask::onTaskEndSuccess: Task b9c76de6-a89c-47d2-9e54-a4b936417438 (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 12:29:56,978 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-84) CommandAsyncTask::endActionIfNecessary: All tasks of command 84464c96-fedf-477b-900e-7c61c7aa1a47 has ended -> executing endAction >2014-08-13 12:29:56,978 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-84) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 84464c96-fedf-477b-900e-7c61c7aa1a47): calling endAction . >2014-08-13 12:29:56,979 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-28) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddVmFromSnapshot, executionIndex: 0 >2014-08-13 12:29:57,010 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (org.ovirt.thread.pool-8-thread-28) [30619db6] Ending command successfully: org.ovirt.engine.core.bll.AddVmFromSnapshotCommand >2014-08-13 12:29:57,043 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-8-thread-28) [13066325] Ending command successfully: org.ovirt.engine.core.bll.CopyImageGroupCommand >2014-08-13 12:29:57,047 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-28) [13066325] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 2d973705-7786-428d-ab98-25201dbcd8c7, imageId = 0c77e8c1-8f7a-4754-be5d-7489a96c704e), log id: 614d0f81 >2014-08-13 12:29:57,314 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-28) [13066325] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@8bc02daf, log id: 614d0f81 >2014-08-13 12:29:57,317 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-8-thread-28) START, SetVmStatusVDSCommand( vmId = c7e040ec-6614-4600-9154-594127fdf837, status = Down, exit status = Normal), log id: 2c094a3 >2014-08-13 12:29:57,320 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-8-thread-28) FINISH, SetVmStatusVDSCommand, log id: 2c094a3 >2014-08-13 12:29:57,367 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-28) Correlation ID: 30619db6, Call Stack: null, Custom Event ID: -1, Message: VM ggg creation has been completed. >2014-08-13 12:29:57,368 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-28) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVmFromSnapshot completed, handling the result. >2014-08-13 12:29:57,368 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-28) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddVmFromSnapshot succeeded, clearing tasks. >2014-08-13 12:29:57,375 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-28) SPMAsyncTask::ClearAsyncTask: Attempting to clear task b9c76de6-a89c-47d2-9e54-a4b936417438 >2014-08-13 12:29:57,376 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-28) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = b9c76de6-a89c-47d2-9e54-a4b936417438), log id: f8643b6 >2014-08-13 12:29:57,378 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-28) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=b9c76de6-a89c-47d2-9e54-a4b936417438), log id: 37c769b0 >2014-08-13 12:29:57,703 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-28) FINISH, HSMClearTaskVDSCommand, log id: 37c769b0 >2014-08-13 12:29:57,704 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-28) FINISH, SPMClearTaskVDSCommand, log id: f8643b6 >2014-08-13 12:29:57,764 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-28) BaseAsyncTask::removeTaskFromDB: Removed task b9c76de6-a89c-47d2-9e54-a4b936417438 from DataBase >2014-08-13 12:29:57,765 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-28) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 84464c96-fedf-477b-900e-7c61c7aa1a47 >2014-08-13 12:31:01,334 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-69) Setting new tasks map. The map contains now 0 tasks >2014-08-13 12:31:01,334 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-69) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-13 12:55:53,451 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-59) Principal admin::internal synchronized >2014-08-13 12:55:53,758 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-6) [69594bd8] Attempting to update VMs/Templates Ovf. >2014-08-13 12:55:54,131 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-6) [368b44b4] Lock Acquired to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 12:55:54,134 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-6) [368b44b4] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-13 12:55:54,210 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [368b44b4] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4c16b586-1b20-4f3a-9001-43521e3bb552, imageId = 655fad6a-3526-407e-a7d9-2d5e09102565), log id: 64e5bac6 >2014-08-13 12:55:54,210 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [368b44b4] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 12:55:54,211 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [368b44b4] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4c16b586-1b20-4f3a-9001-43521e3bb552 > volUUID=655fad6a-3526-407e-a7d9-2d5e09102565 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Tue Aug 12 15:59:03 IDT 2014"} > >2014-08-13 12:55:54,485 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [368b44b4] FINISH, SetVolumeDescriptionVDSCommand, log id: 64e5bac6 >2014-08-13 12:55:54,537 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] Running command: UploadStreamCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-13 12:55:54,543 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 29f6d267 >2014-08-13 12:55:54,543 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] -- executeVdsBrokerCommand: >2014-08-13 12:55:54,544 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGUID=4c16b586-1b20-4f3a-9001-43521e3bb552 > volUUID=655fad6a-3526-407e-a7d9-2d5e09102565 > size=71,680 > >2014-08-13 12:55:57,649 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] FINISH, UploadStreamVDSCommand, log id: 29f6d267 >2014-08-13 12:55:57,654 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-6) [ded2320] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 765cdbbb-5aa4-4f29-adbf-f14127156323 >2014-08-13 12:55:57,655 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-6) [ded2320] CommandMultiAsyncTasks::AttachTask: Attaching task 4488a00c-220a-4897-9050-0a20f04f466a to command 765cdbbb-5aa4-4f29-adbf-f14127156323. >2014-08-13 12:55:57,726 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-6) [ded2320] Adding task 4488a00c-220a-4897-9050-0a20f04f466a (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 12:55:57,743 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 12:55:57,746 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4c16b586-1b20-4f3a-9001-43521e3bb552, imageId = 655fad6a-3526-407e-a7d9-2d5e09102565), log id: 6e02a24a >2014-08-13 12:55:57,747 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 12:55:57,748 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4c16b586-1b20-4f3a-9001-43521e3bb552 > volUUID=655fad6a-3526-407e-a7d9-2d5e09102565 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Wed Aug 13 12:55:54 IDT 2014","Size":71680} > >2014-08-13 12:55:57,996 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] FINISH, SetVolumeDescriptionVDSCommand, log id: 6e02a24a >2014-08-13 12:55:58,010 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4cab3024-3f6d-435a-bf22-a6e7daac3bd9, imageId = 612a0f89-926c-48c5-9804-b258314e4838), log id: 60d0e3db >2014-08-13 12:55:58,011 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 12:55:58,012 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4cab3024-3f6d-435a-bf22-a6e7daac3bd9 > volUUID=612a0f89-926c-48c5-9804-b258314e4838 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Tue Aug 12 15:59:03 IDT 2014"} > >2014-08-13 12:55:58,302 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [ded2320] FINISH, SetVolumeDescriptionVDSCommand, log id: 60d0e3db >2014-08-13 12:55:58,347 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-6) [2878b9aa] Running command: UploadStreamCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-13 12:55:58,349 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [2878b9aa] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 5e3d31c6 >2014-08-13 12:55:58,349 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [2878b9aa] -- executeVdsBrokerCommand: >2014-08-13 12:55:58,350 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [2878b9aa] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGUID=4cab3024-3f6d-435a-bf22-a6e7daac3bd9 > volUUID=612a0f89-926c-48c5-9804-b258314e4838 > size=71,680 > >2014-08-13 12:56:02,762 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [2878b9aa] FINISH, UploadStreamVDSCommand, log id: 5e3d31c6 >2014-08-13 12:56:02,765 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-6) [2878b9aa] CommandMultiAsyncTasks::AttachTask: Attaching task 3b222f6b-6c93-47eb-9897-08605ca1f684 to command 765cdbbb-5aa4-4f29-adbf-f14127156323. >2014-08-13 12:56:02,862 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-6) [2878b9aa] Adding task 3b222f6b-6c93-47eb-9897-08605ca1f684 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 12:56:02,878 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-6) [2878b9aa] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-13 12:56:02,879 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-6) [2878b9aa] Trying to release exclusive lock which does not exist, lock key: d54e6e6c-81f2-41f2-90ab-7e3c2457938aSTORAGE >2014-08-13 12:56:02,879 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-6) [2878b9aa] Trying to release exclusive lock which does not exist, lock key: 4c16b586-1b20-4f3a-9001-43521e3bb552DISK >2014-08-13 12:56:02,880 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-6) [2878b9aa] Trying to release exclusive lock which does not exist, lock key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9DISK >2014-08-13 12:56:02,880 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-6) [2878b9aa] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 12:56:02,886 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [2878b9aa] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4cab3024-3f6d-435a-bf22-a6e7daac3bd9, imageId = 612a0f89-926c-48c5-9804-b258314e4838), log id: 2ecf3ccf >2014-08-13 12:56:02,887 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [2878b9aa] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 12:56:02,888 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [2878b9aa] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4cab3024-3f6d-435a-bf22-a6e7daac3bd9 > volUUID=612a0f89-926c-48c5-9804-b258314e4838 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Wed Aug 13 12:55:54 IDT 2014","Size":71680} > >2014-08-13 12:56:03,096 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [2878b9aa] FINISH, SetVolumeDescriptionVDSCommand, log id: 2ecf3ccf >2014-08-13 12:56:03,102 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-6) [2878b9aa] BaseAsyncTask::startPollingTask: Starting to poll task 4488a00c-220a-4897-9050-0a20f04f466a. >2014-08-13 12:56:03,103 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-6) [2878b9aa] BaseAsyncTask::startPollingTask: Starting to poll task 3b222f6b-6c93-47eb-9897-08605ca1f684. >2014-08-13 12:56:03,118 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-6) [7918dc5b] Lock Acquired to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 12:56:03,120 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-6) [7918dc5b] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-13 12:56:03,127 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [7918dc5b] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 3867fc28 >2014-08-13 12:56:03,128 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [7918dc5b] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 12:56:03,128 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [7918dc5b] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Tue Aug 12 14:58:59 IDT 2014"} > >2014-08-13 12:56:03,506 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [7918dc5b] FINISH, SetVolumeDescriptionVDSCommand, log id: 3867fc28 >2014-08-13 12:56:03,557 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-13 12:56:03,559 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 63931479 >2014-08-13 12:56:03,559 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] -- executeVdsBrokerCommand: >2014-08-13 12:56:03,560 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > size=81,920 > >2014-08-13 12:56:06,736 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] FINISH, UploadStreamVDSCommand, log id: 63931479 >2014-08-13 12:56:06,739 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-6) [43560bd3] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 6cc8852e-0339-4ab2-8f44-4862f9512a03 >2014-08-13 12:56:06,740 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-6) [43560bd3] CommandMultiAsyncTasks::AttachTask: Attaching task 56cdb595-f2f2-4fd1-9735-aaea5333605e to command 6cc8852e-0339-4ab2-8f44-4862f9512a03. >2014-08-13 12:56:06,787 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-6) [43560bd3] Adding task 56cdb595-f2f2-4fd1-9735-aaea5333605e (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 12:56:06,804 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 12:56:06,812 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 6fa63adc >2014-08-13 12:56:06,813 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 12:56:06,814 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Wed Aug 13 12:56:03 IDT 2014","Size":81920} > >2014-08-13 12:56:07,024 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-79) Polling and updating Async Tasks: 3 tasks, 2 tasks to poll now >2014-08-13 12:56:07,161 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] FINISH, SetVolumeDescriptionVDSCommand, log id: 6fa63adc >2014-08-13 12:56:07,169 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 732c6b43 >2014-08-13 12:56:07,971 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 12:56:07,971 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) SPMAsyncTask::PollTask: Polling task 3b222f6b-6c93-47eb-9897-08605ca1f684 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 12:56:07,972 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Tue Aug 12 14:58:59 IDT 2014"} > >2014-08-13 12:56:07,973 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) BaseAsyncTask::onTaskEndSuccess: Task 3b222f6b-6c93-47eb-9897-08605ca1f684 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 12:56:07,975 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-79) Task with DB Task ID bbcb70a0-ee93-49a8-943c-017b98fcd34a and VDSM Task ID 4488a00c-220a-4897-9050-0a20f04f466a is in state Polling. End action for command 765cdbbb-5aa4-4f29-adbf-f14127156323 will proceed when all the entitys tasks are completed. >2014-08-13 12:56:07,976 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) SPMAsyncTask::PollTask: Polling task 4488a00c-220a-4897-9050-0a20f04f466a (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 12:56:07,977 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-79) BaseAsyncTask::onTaskEndSuccess: Task 4488a00c-220a-4897-9050-0a20f04f466a (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 12:56:07,978 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-79) CommandAsyncTask::endActionIfNecessary: All tasks of command 765cdbbb-5aa4-4f29-adbf-f14127156323 has ended -> executing endAction >2014-08-13 12:56:07,979 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-79) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 765cdbbb-5aa4-4f29-adbf-f14127156323): calling endAction . >2014-08-13 12:56:07,980 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-13) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-08-13 12:56:08,028 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-13) [368b44b4] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-08-13 12:56:08,029 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-13) [368b44b4] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-13 12:56:08,029 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-13) [368b44b4] Trying to release exclusive lock which does not exist, lock key: d54e6e6c-81f2-41f2-90ab-7e3c2457938aSTORAGE >2014-08-13 12:56:08,030 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-13) [368b44b4] Trying to release exclusive lock which does not exist, lock key: 4c16b586-1b20-4f3a-9001-43521e3bb552DISK >2014-08-13 12:56:08,030 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-13) [368b44b4] Trying to release exclusive lock which does not exist, lock key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9DISK >2014-08-13 12:56:08,031 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-13) [368b44b4] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 12:56:08,031 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-13) [368b44b4] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-08-13 12:56:08,032 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-13) [368b44b4] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-08-13 12:56:08,032 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-13) [368b44b4] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 3b222f6b-6c93-47eb-9897-08605ca1f684 >2014-08-13 12:56:08,033 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) [368b44b4] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 3b222f6b-6c93-47eb-9897-08605ca1f684), log id: 359b240 >2014-08-13 12:56:08,397 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [43560bd3] FINISH, SetVolumeDescriptionVDSCommand, log id: 732c6b43 >2014-08-13 12:56:08,398 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) [368b44b4] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=3b222f6b-6c93-47eb-9897-08605ca1f684), log id: 75c58977 >2014-08-13 12:56:08,452 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-6) [78e54d3f] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-13 12:56:08,453 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [78e54d3f] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 48e4de3e >2014-08-13 12:56:08,453 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [78e54d3f] -- executeVdsBrokerCommand: >2014-08-13 12:56:08,454 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [78e54d3f] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > size=81,920 > >2014-08-13 12:56:08,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) [368b44b4] FINISH, HSMClearTaskVDSCommand, log id: 75c58977 >2014-08-13 12:56:08,732 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) [368b44b4] FINISH, SPMClearTaskVDSCommand, log id: 359b240 >2014-08-13 12:56:08,750 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-13) [368b44b4] BaseAsyncTask::removeTaskFromDB: Removed task 3b222f6b-6c93-47eb-9897-08605ca1f684 from DataBase >2014-08-13 12:56:08,750 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-13) [368b44b4] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 4488a00c-220a-4897-9050-0a20f04f466a >2014-08-13 12:56:08,751 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) [368b44b4] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 4488a00c-220a-4897-9050-0a20f04f466a), log id: 62a431e3 >2014-08-13 12:56:08,753 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) [368b44b4] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=4488a00c-220a-4897-9050-0a20f04f466a), log id: 9c760ad >2014-08-13 12:56:09,161 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) [368b44b4] FINISH, HSMClearTaskVDSCommand, log id: 9c760ad >2014-08-13 12:56:09,162 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) [368b44b4] FINISH, SPMClearTaskVDSCommand, log id: 62a431e3 >2014-08-13 12:56:09,181 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-13) [368b44b4] BaseAsyncTask::removeTaskFromDB: Removed task 4488a00c-220a-4897-9050-0a20f04f466a from DataBase >2014-08-13 12:56:09,181 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-13) [368b44b4] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 765cdbbb-5aa4-4f29-adbf-f14127156323 >2014-08-13 12:56:11,443 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-6) [78e54d3f] FINISH, UploadStreamVDSCommand, log id: 48e4de3e >2014-08-13 12:56:11,446 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-6) [78e54d3f] CommandMultiAsyncTasks::AttachTask: Attaching task 9adb38c4-831d-4568-b01f-d040f83902d5 to command 6cc8852e-0339-4ab2-8f44-4862f9512a03. >2014-08-13 12:56:11,492 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-6) [78e54d3f] Adding task 9adb38c4-831d-4568-b01f-d040f83902d5 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 12:56:11,508 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-6) [78e54d3f] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-13 12:56:11,508 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-6) [78e54d3f] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-13 12:56:11,509 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-6) [78e54d3f] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-13 12:56:11,509 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-6) [78e54d3f] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-13 12:56:11,510 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-6) [78e54d3f] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 12:56:11,511 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [78e54d3f] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: b12cd53 >2014-08-13 12:56:11,512 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [78e54d3f] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 12:56:11,512 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [78e54d3f] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Wed Aug 13 12:56:03 IDT 2014","Size":81920} > >2014-08-13 12:56:11,815 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-6) [78e54d3f] FINISH, SetVolumeDescriptionVDSCommand, log id: b12cd53 >2014-08-13 12:56:11,823 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-6) [78e54d3f] BaseAsyncTask::startPollingTask: Starting to poll task 56cdb595-f2f2-4fd1-9735-aaea5333605e. >2014-08-13 12:56:11,823 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-6) [78e54d3f] BaseAsyncTask::startPollingTask: Starting to poll task 9adb38c4-831d-4568-b01f-d040f83902d5. >2014-08-13 12:56:17,981 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-55) Polling and updating Async Tasks: 4 tasks, 2 tasks to poll now >2014-08-13 12:56:18,273 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) SPMAsyncTask::PollTask: Polling task 9adb38c4-831d-4568-b01f-d040f83902d5 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 12:56:18,274 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) BaseAsyncTask::onTaskEndSuccess: Task 9adb38c4-831d-4568-b01f-d040f83902d5 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 12:56:18,275 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-55) Task with DB Task ID 18b61b06-30d0-444b-baf4-dacd1ffd454c and VDSM Task ID 56cdb595-f2f2-4fd1-9735-aaea5333605e is in state Polling. End action for command 6cc8852e-0339-4ab2-8f44-4862f9512a03 will proceed when all the entitys tasks are completed. >2014-08-13 12:56:18,276 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) SPMAsyncTask::PollTask: Polling task 56cdb595-f2f2-4fd1-9735-aaea5333605e (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 12:56:18,278 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) BaseAsyncTask::onTaskEndSuccess: Task 56cdb595-f2f2-4fd1-9735-aaea5333605e (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 12:56:18,279 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-55) CommandAsyncTask::endActionIfNecessary: All tasks of command 6cc8852e-0339-4ab2-8f44-4862f9512a03 has ended -> executing endAction >2014-08-13 12:56:18,280 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-55) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 6cc8852e-0339-4ab2-8f44-4862f9512a03): calling endAction . >2014-08-13 12:56:18,281 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-17) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-08-13 12:56:18,302 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-08-13 12:56:18,302 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-13 12:56:18,303 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-13 12:56:18,303 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-13 12:56:18,304 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-13 12:56:18,304 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 12:56:18,305 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-08-13 12:56:18,305 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-08-13 12:56:18,306 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 9adb38c4-831d-4568-b01f-d040f83902d5 >2014-08-13 12:56:18,307 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 9adb38c4-831d-4568-b01f-d040f83902d5), log id: 4b023661 >2014-08-13 12:56:18,308 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=9adb38c4-831d-4568-b01f-d040f83902d5), log id: 7e4040bf >2014-08-13 12:56:18,716 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] FINISH, HSMClearTaskVDSCommand, log id: 7e4040bf >2014-08-13 12:56:18,717 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] FINISH, SPMClearTaskVDSCommand, log id: 4b023661 >2014-08-13 12:56:18,764 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] BaseAsyncTask::removeTaskFromDB: Removed task 9adb38c4-831d-4568-b01f-d040f83902d5 from DataBase >2014-08-13 12:56:18,764 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 56cdb595-f2f2-4fd1-9735-aaea5333605e >2014-08-13 12:56:18,765 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 56cdb595-f2f2-4fd1-9735-aaea5333605e), log id: 20742b55 >2014-08-13 12:56:18,767 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=56cdb595-f2f2-4fd1-9735-aaea5333605e), log id: 2ac8ea18 >2014-08-13 12:56:19,140 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] FINISH, HSMClearTaskVDSCommand, log id: 2ac8ea18 >2014-08-13 12:56:19,141 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] FINISH, SPMClearTaskVDSCommand, log id: 20742b55 >2014-08-13 12:56:19,153 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] BaseAsyncTask::removeTaskFromDB: Removed task 56cdb595-f2f2-4fd1-9735-aaea5333605e from DataBase >2014-08-13 12:56:19,154 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-17) [7918dc5b] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 6cc8852e-0339-4ab2-8f44-4862f9512a03 >2014-08-13 12:57:31,355 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-39) Setting new tasks map. The map contains now 0 tasks >2014-08-13 12:57:31,356 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-39) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-13 13:44:21,336 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-2) Running command: LoginUserCommand internal: false. >2014-08-13 13:44:21,349 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-13 13:44:21,481 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-2) [6426229b] Running command: LogoutBySessionCommand internal: false. >2014-08-13 13:44:21,486 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-2) [1ddeaf75] Running command: LogoutUserCommand internal: false. >2014-08-13 13:44:21,498 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [1ddeaf75] Correlation ID: 1ddeaf75, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-13 13:44:21,607 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-2) Running command: LoginUserCommand internal: false. >2014-08-13 13:44:21,682 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-2) [2776d1ac] Running command: LogoutBySessionCommand internal: false. >2014-08-13 13:44:21,684 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-2) [56c599a2] Running command: LogoutUserCommand internal: false. >2014-08-13 13:44:21,713 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [56c599a2] Correlation ID: 56c599a2, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-13 13:44:21,757 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-2) Running command: LoginUserCommand internal: false. >2014-08-13 13:44:21,819 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-2) [38ef9329] Running command: LogoutBySessionCommand internal: false. >2014-08-13 13:44:21,822 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-2) [463f9c63] Running command: LogoutUserCommand internal: false. >2014-08-13 13:44:21,829 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [463f9c63] Correlation ID: 463f9c63, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-13 13:45:14,695 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-2) Running command: LoginUserCommand internal: false. >2014-08-13 13:45:14,701 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-13 13:45:14,771 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-2) [642ace1e] Running command: LogoutBySessionCommand internal: false. >2014-08-13 13:45:14,772 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-2) [1bee86e] Running command: LogoutUserCommand internal: false. >2014-08-13 13:45:14,784 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [1bee86e] Correlation ID: 1bee86e, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-13 13:45:14,927 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-2) Running command: LoginUserCommand internal: false. >2014-08-13 13:45:14,965 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-2) [7070e4b3] Running command: LogoutBySessionCommand internal: false. >2014-08-13 13:45:14,967 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-2) [a814103] Running command: LogoutUserCommand internal: false. >2014-08-13 13:45:14,975 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [a814103] Correlation ID: a814103, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-13 13:45:47,704 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-5) Running command: LoginUserCommand internal: false. >2014-08-13 13:45:47,709 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-13 13:45:47,773 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-5) [6825ba03] Running command: LogoutBySessionCommand internal: false. >2014-08-13 13:45:47,776 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-5) [565b5e94] Running command: LogoutUserCommand internal: false. >2014-08-13 13:45:47,792 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-5) [565b5e94] Correlation ID: 565b5e94, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-13 13:48:21,936 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-2) Running command: LoginUserCommand internal: false. >2014-08-13 13:48:21,949 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-13 13:48:21,964 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-2) [76b6667] Running command: LogoutBySessionCommand internal: false. >2014-08-13 13:48:21,967 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-2) [3fc7e42e] Running command: LogoutUserCommand internal: false. >2014-08-13 13:48:21,974 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [3fc7e42e] Correlation ID: 3fc7e42e, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-13 13:48:58,554 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-2) Running command: LoginUserCommand internal: false. >2014-08-13 13:48:58,560 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-13 13:48:58,656 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (http--0.0.0.0-8080-2) [7ca0bf68] START, GetImagesListVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, sdUUID = d54e6e6c-81f2-41f2-90ab-7e3c2457938a), log id: 5d4e6d63 >2014-08-13 13:48:58,917 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (http--0.0.0.0-8080-2) [7ca0bf68] FINISH, GetImagesListVDSCommand, return: [2d973705-7786-428d-ab98-25201dbcd8c7, 4cab3024-3f6d-435a-bf22-a6e7daac3bd9, 1560ea5f-3a4d-4810-a071-6cf6a1c7d898, 4c16b586-1b20-4f3a-9001-43521e3bb552, 2b033ab4-8cd6-4214-baf6-1252264f371e, 8d0f851c-38be-4d3e-a278-4af48a16689a, 016a7ddb-9007-4925-ae72-202649686c94, 13d602b9-2253-4fd0-9409-813c8007cf8f, 78dde0c0-a5d7-4e8c-bfe7-de91d182d512], log id: 5d4e6d63 >2014-08-13 13:48:58,919 WARN [org.ovirt.engine.core.bll.MoveVmCommand] (http--0.0.0.0-8080-2) [7ca0bf68] CanDoAction of action MoveVm failed. Reasons:VAR__ACTION__MOVE,VAR__TYPE__VM,ACTION_TYPE_FAILED_STORAGE_DOMAIN_ALREADY_CONTAINS_DISK >2014-08-13 13:48:58,934 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (http--0.0.0.0-8080-2) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-13 13:48:58,939 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (http--0.0.0.0-8080-2) Operation Failed: [Cannot move VM. The Storage Domain already contains the target disk(s).] >2014-08-13 13:48:58,942 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-2) [f2119b3] Running command: LogoutBySessionCommand internal: false. >2014-08-13 13:48:58,946 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-2) [53642e69] Running command: LogoutUserCommand internal: false. >2014-08-13 13:48:58,958 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [53642e69] Correlation ID: 53642e69, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-13 13:50:12,488 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-2) Running command: LoginUserCommand internal: false. >2014-08-13 13:50:12,520 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-13 13:50:12,591 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (http--0.0.0.0-8080-2) [4df7bd83] START, GetImagesListVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, sdUUID = 7158eae9-4886-4af7-af0b-d668ecee9e4f), log id: 1c453b1c >2014-08-13 13:50:12,748 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (http--0.0.0.0-8080-2) [4df7bd83] FINISH, GetImagesListVDSCommand, return: [d98ca030-8e70-4ea9-88ea-195101cf42de, 34feecc4-b988-4733-94f6-5debb661f0b6, 4c1a2b29-52c6-48f2-b3e9-d3dbfb96d7b1, b5576e45-7f13-465f-b26f-05d52faf28d1, 6b0f7b39-7b10-49e9-9d5b-44efa62ed7a8, 4b87d5b7-4bd3-4b27-b9c3-6582222622be, ee28320c-7b27-4dfe-a4dc-a00f523ed516, 9bbaea85-87de-4005-9f80-7dbc97cff94a, 056639f7-5123-4146-b4f5-49c9284f7ff6, 3072eb0e-6fe8-471f-b422-e1f86205a232, 99e5cda5-262b-4469-b336-04e3f0cf7c38, 4b0c7abe-d9b3-4b7f-b5ec-407fa9a7eb0e, af890a8b-f1cc-40c4-959c-3c3bf08008a1, 5701f63e-9b85-4238-8370-2d791c25287e, 4a8599dd-7743-4491-9a6f-3d3931d447bd, db02b9cf-de9a-438a-9983-3549876b1e39, 5e77c76a-ab0c-4dab-b1cc-408e6e54f248, 4d058772-bb18-4f52-bb99-4d7601152ae0, 9a19a230-ca3a-41c0-9b3f-4483d2f4895b, 8873ab86-8adc-43d0-82f3-97379cb93872, f6edaf20-e507-471c-b6c1-487c35e1d7cf, 30b19e8d-633c-4cc8-b5a4-61e506e7bed9, 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, a847794e-193a-45f6-a62b-5f0eba8a4660, fd73926f-e6f9-4892-8c00-a0e089d7a7f0], log id: 1c453b1c >2014-08-13 13:56:19,399 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to update VMs/Templates Ovf. >2014-08-13 13:56:24,004 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-53) Command SpmStatusVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException >2014-08-13 13:56:24,010 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-35) Principal admin::internal synchronized >2014-08-13 13:56:24,045 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-53) [132d65cc] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 13:56:24,064 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-53) [132d65cc] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-13 13:56:24,074 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-53) [132d65cc] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-13 13:57:22,355 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) [132d65cc] Correlation ID: 132d65cc, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host h1, Error: Network error during communication with the Host.). >2014-08-13 13:57:40,261 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-10) Command ListVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: org.apache.commons.httpclient.NoHttpResponseException: The server battlestar.tlv.redhat.com failed to respond >2014-08-13 13:57:40,262 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of $81 seconds and after that an attempt to fence the host will be issued. >2014-08-13 13:57:40,272 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.commons.httpclient.NoHttpResponseException: The server battlestar.tlv.redhat.com failed to respond > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:407) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.fetchRunningVms(VdsUpdateRunTimeInfo.java:971) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:920) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:638) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.apache.commons.httpclient.NoHttpResponseException: The server battlestar.tlv.redhat.com failed to respond > at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1976) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.list(Unknown Source) > at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-13 13:57:42,599 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >org.apache.commons.httpclient.NoHttpResponseException: The server battlestar.tlv.redhat.com failed to respond >2014-08-13 13:58:20,772 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-41) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 212b5219 >2014-08-13 13:58:29,725 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-41) FINISH, GetHardwareInfoVDSCommand, log id: 212b5219 >2014-08-13 13:58:31,273 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-13 13:58:31,274 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Host h1 is running with disabled SELinux. >2014-08-13 13:58:31,294 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-32) Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : no reporting hosts >2014-08-13 13:58:31,362 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-32) Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : no reporting hosts >2014-08-13 13:58:31,376 INFO [org.ovirt.engine.core.bll.MoveVmCommand] (http--0.0.0.0-8080-2) [4df7bd83] Running command: MoveVmCommand internal: false. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: StorageAction group MOVE_VM with role type USER >2014-08-13 13:58:31,378 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [4df7bd83] START, IsVmDuringInitiatingVDSCommand( vmId = c7e040ec-6614-4600-9154-594127fdf837), log id: 6c28064e >2014-08-13 13:58:31,379 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-2) [4df7bd83] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 6c28064e >2014-08-13 13:58:31,380 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (http--0.0.0.0-8080-2) [4df7bd83] START, SetVmStatusVDSCommand( vmId = c7e040ec-6614-4600-9154-594127fdf837, status = ImageLocked, exit status = Normal), log id: 1689a0f0 >2014-08-13 13:58:31,399 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (http--0.0.0.0-8080-2) [4df7bd83] FINISH, SetVmStatusVDSCommand, log id: 1689a0f0 >2014-08-13 13:58:31,414 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-41) [2cb9fce7] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-13 13:58:31,434 INFO [org.ovirt.engine.core.bll.MoveImageGroupCommand] (http--0.0.0.0-8080-2) [7356f0da] Running command: MoveImageGroupCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-13 13:58:31,508 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.MoveImageGroupVDSCommand] (http--0.0.0.0-8080-2) [7356f0da] START, MoveImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 2d973705-7786-428d-ab98-25201dbcd8c7, dstDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, vmId = c7e040ec-6614-4600-9154-594127fdf837, op = Copy, postZero = false, force = false), log id: 1d997de9 >2014-08-13 13:58:32,616 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-41) [1c521cfb] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 13:58:32,715 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-41) [15df5581] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-13 13:58:32,724 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-41) [15df5581] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = NFS, connectionList = [{ id: 1670ace3-7db6-4c73-9b3c-614dcc85bd46, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc5, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc4, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 5bdea370 >2014-08-13 13:58:32,797 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.MoveImageGroupVDSCommand] (http--0.0.0.0-8080-2) [7356f0da] FINISH, MoveImageGroupVDSCommand, log id: 1d997de9 >2014-08-13 13:58:32,824 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (http--0.0.0.0-8080-2) [7356f0da] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command adc74f11-e721-47f5-9209-23ddbf142047 >2014-08-13 13:58:32,824 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (http--0.0.0.0-8080-2) [7356f0da] CommandMultiAsyncTasks::AttachTask: Attaching task 2d136784-d6d4-43a8-9806-e332c4a97556 to command adc74f11-e721-47f5-9209-23ddbf142047. >2014-08-13 13:58:32,852 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (http--0.0.0.0-8080-2) [7356f0da] Adding task 2d136784-d6d4-43a8-9806-e332c4a97556 (Parent Command MoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 13:58:32,919 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [7356f0da] Correlation ID: 4df7bd83, Job ID: 5541ecbb-f56e-45e2-acf5-033f639f3f3b, Call Stack: null, Custom Event ID: -1, Message: VM ggg moving to Domain sd35_1 was initiated by admin. >2014-08-13 13:58:32,919 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (http--0.0.0.0-8080-2) [7356f0da] BaseAsyncTask::startPollingTask: Starting to poll task 2d136784-d6d4-43a8-9806-e332c4a97556. >2014-08-13 13:58:32,924 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (http--0.0.0.0-8080-2) [66de9cd8] Running command: LogoutBySessionCommand internal: false. >2014-08-13 13:58:32,927 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-2) [6be30d18] Running command: LogoutUserCommand internal: false. >2014-08-13 13:58:32,943 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [6be30d18] Correlation ID: 6be30d18, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-13 13:58:34,181 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-41) [15df5581] FINISH, ConnectStorageServerVDSCommand, return: {1670ace3-7db6-4c73-9b3c-614dcc85bd46=0, 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a=0}, log id: 5bdea370 >2014-08-13 13:58:34,184 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-41) [15df5581] Host h1 storage connection was succeeded >2014-08-13 13:58:34,208 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-13) START, ConnectStoragePoolVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vdsId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, masterVersion = 1), log id: 5eda9384 >2014-08-13 13:58:35,534 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-13) FINISH, ConnectStoragePoolVDSCommand, log id: 5eda9384 >2014-08-13 13:58:37,446 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-41) [15df5581] START, SetMOMPolicyParametersVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 488376f >2014-08-13 13:58:37,776 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-41) [15df5581] FINISH, SetMOMPolicyParametersVDSCommand, log id: 488376f >2014-08-13 13:58:37,813 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) [15df5581] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host h1 was set to Up. >2014-08-13 13:58:37,819 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) [15df5581] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h1. >2014-08-13 13:58:37,847 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-41) [198688c7] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-13 13:58:41,273 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-77) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-13 13:58:41,579 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-77) SPMAsyncTask::PollTask: Polling task 2d136784-d6d4-43a8-9806-e332c4a97556 (Parent Command MoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-13 13:58:41,579 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-77) Finished polling Tasks, will poll again in 10 seconds. >2014-08-13 14:10:10,674 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-100) [69a1e809] SPMAsyncTask::PollTask: Polling task 2d136784-d6d4-43a8-9806-e332c4a97556 (Parent Command MoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 14:10:10,690 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-100) [69a1e809] BaseAsyncTask::onTaskEndSuccess: Task 2d136784-d6d4-43a8-9806-e332c4a97556 (Parent Command MoveVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 14:10:10,691 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-100) [69a1e809] CommandAsyncTask::endActionIfNecessary: All tasks of command adc74f11-e721-47f5-9209-23ddbf142047 has ended -> executing endAction >2014-08-13 14:10:10,692 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-100) [69a1e809] CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: adc74f11-e721-47f5-9209-23ddbf142047): calling endAction . >2014-08-13 14:10:10,693 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-20) [69a1e809] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction MoveVm, executionIndex: 0 >2014-08-13 14:10:10,740 INFO [org.ovirt.engine.core.bll.MoveVmCommand] (org.ovirt.thread.pool-8-thread-20) [4df7bd83] Ending command successfully: org.ovirt.engine.core.bll.MoveVmCommand >2014-08-13 14:10:10,756 INFO [org.ovirt.engine.core.bll.MoveImageGroupCommand] (org.ovirt.thread.pool-8-thread-20) [7356f0da] Ending command successfully: org.ovirt.engine.core.bll.MoveImageGroupCommand >2014-08-13 14:10:10,789 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-20) [1388d07d] Running command: RemoveImageCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-13 14:10:10,815 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-20) [1388d07d] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 2d973705-7786-428d-ab98-25201dbcd8c7, postZeros = false, forceDelete = false), log id: 1ccf5282 >2014-08-13 14:10:12,382 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-8-thread-20) [1388d07d] FINISH, DeleteImageGroupVDSCommand, log id: 1ccf5282 >2014-08-13 14:10:12,392 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-20) [1388d07d] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command e75446b1-0101-41ae-b776-eada2917473d >2014-08-13 14:10:12,393 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-20) [1388d07d] CommandMultiAsyncTasks::AttachTask: Attaching task 6a97c3f6-0d81-427a-a6fd-3be77afbff15 to command e75446b1-0101-41ae-b776-eada2917473d. >2014-08-13 14:10:12,446 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-20) [1388d07d] Adding task 6a97c3f6-0d81-427a-a6fd-3be77afbff15 (Parent Command RemoveImage, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 14:10:12,465 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-20) [1388d07d] BaseAsyncTask::startPollingTask: Starting to poll task 6a97c3f6-0d81-427a-a6fd-3be77afbff15. >2014-08-13 14:10:12,474 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-20) [1388d07d] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 2d973705-7786-428d-ab98-25201dbcd8c7, imageId = 0c77e8c1-8f7a-4754-be5d-7489a96c704e), log id: 46656e79 >2014-08-13 14:10:12,781 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-20) [1388d07d] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@9d437700, log id: 46656e79 >2014-08-13 14:10:12,785 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-8-thread-20) START, SetVmStatusVDSCommand( vmId = c7e040ec-6614-4600-9154-594127fdf837, status = Down, exit status = Normal), log id: 6505d076 >2014-08-13 14:10:12,787 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-8-thread-20) FINISH, SetVmStatusVDSCommand, log id: 6505d076 >2014-08-13 14:10:12,827 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-20) Correlation ID: 4df7bd83, Call Stack: null, Custom Event ID: -1, Message: Moving VM ggg to Domain sd35_1 has been completed. >2014-08-13 14:10:12,828 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-20) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type MoveVm completed, handling the result. >2014-08-13 14:10:12,828 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-20) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type MoveVm succeeded, clearing tasks. >2014-08-13 14:10:12,836 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-20) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 2d136784-d6d4-43a8-9806-e332c4a97556 >2014-08-13 14:10:12,836 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-20) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 2d136784-d6d4-43a8-9806-e332c4a97556), log id: 1d8a667d >2014-08-13 14:10:12,838 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-20) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=2d136784-d6d4-43a8-9806-e332c4a97556), log id: 694ea7b1 >2014-08-13 14:10:13,141 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-20) FINISH, HSMClearTaskVDSCommand, log id: 694ea7b1 >2014-08-13 14:10:13,142 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-20) FINISH, SPMClearTaskVDSCommand, log id: 1d8a667d >2014-08-13 14:10:13,159 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-20) BaseAsyncTask::removeTaskFromDB: Removed task 2d136784-d6d4-43a8-9806-e332c4a97556 from DataBase >2014-08-13 14:10:13,159 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-20) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity adc74f11-e721-47f5-9209-23ddbf142047 >2014-08-13 14:10:20,694 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-74) Polling and updating Async Tasks: 2 tasks, 1 tasks to poll now >2014-08-13 14:10:21,057 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-74) SPMAsyncTask::PollTask: Polling task 6a97c3f6-0d81-427a-a6fd-3be77afbff15 (Parent Command RemoveImage, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 14:10:21,058 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-74) BaseAsyncTask::onTaskEndSuccess: Task 6a97c3f6-0d81-427a-a6fd-3be77afbff15 (Parent Command RemoveImage, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 14:10:21,059 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-74) CommandAsyncTask::endActionIfNecessary: All tasks of command e75446b1-0101-41ae-b776-eada2917473d has ended -> executing endAction >2014-08-13 14:10:21,060 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-74) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: e75446b1-0101-41ae-b776-eada2917473d): calling endAction . >2014-08-13 14:10:21,061 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RemoveImage, executionIndex: 0 >2014-08-13 14:10:21,086 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-8-thread-32) [1388d07d] Ending command successfully: org.ovirt.engine.core.bll.RemoveImageCommand >2014-08-13 14:10:21,088 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) [1388d07d] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveImage completed, handling the result. >2014-08-13 14:10:21,089 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) [1388d07d] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RemoveImage succeeded, clearing tasks. >2014-08-13 14:10:21,089 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-32) [1388d07d] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 6a97c3f6-0d81-427a-a6fd-3be77afbff15 >2014-08-13 14:10:21,090 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-32) [1388d07d] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 6a97c3f6-0d81-427a-a6fd-3be77afbff15), log id: 5f4b3024 >2014-08-13 14:10:21,091 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-32) [1388d07d] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=6a97c3f6-0d81-427a-a6fd-3be77afbff15), log id: 4145bc5c >2014-08-13 14:10:21,485 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-32) [1388d07d] FINISH, HSMClearTaskVDSCommand, log id: 4145bc5c >2014-08-13 14:10:21,486 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-32) [1388d07d] FINISH, SPMClearTaskVDSCommand, log id: 5f4b3024 >2014-08-13 14:10:21,632 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-32) [1388d07d] BaseAsyncTask::removeTaskFromDB: Removed task 6a97c3f6-0d81-427a-a6fd-3be77afbff15 from DataBase >2014-08-13 14:10:21,634 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-32) [1388d07d] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity e75446b1-0101-41ae-b776-eada2917473d >2014-08-13 14:11:29,910 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-23) Setting new tasks map. The map contains now 0 tasks >2014-08-13 14:11:29,911 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-23) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-13 14:56:24,064 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-27) Principal admin::internal synchronized >2014-08-13 14:57:40,273 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-41) [198688c7] Attempting to update VMs/Templates Ovf. >2014-08-13 14:57:40,772 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-41) [519f6d28] Lock Acquired to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 14:57:40,775 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-41) [519f6d28] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-13 14:57:40,829 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [519f6d28] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4c16b586-1b20-4f3a-9001-43521e3bb552, imageId = 655fad6a-3526-407e-a7d9-2d5e09102565), log id: 5c934385 >2014-08-13 14:57:40,831 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [519f6d28] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 14:57:40,832 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [519f6d28] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4c16b586-1b20-4f3a-9001-43521e3bb552 > volUUID=655fad6a-3526-407e-a7d9-2d5e09102565 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Wed Aug 13 12:55:54 IDT 2014"} > >2014-08-13 14:57:41,235 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [519f6d28] FINISH, SetVolumeDescriptionVDSCommand, log id: 5c934385 >2014-08-13 14:57:41,423 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] Running command: UploadStreamCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-13 14:57:41,425 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 7ea3b9bd >2014-08-13 14:57:41,426 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] -- executeVdsBrokerCommand: >2014-08-13 14:57:41,426 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGUID=4c16b586-1b20-4f3a-9001-43521e3bb552 > volUUID=655fad6a-3526-407e-a7d9-2d5e09102565 > size=61,440 > >2014-08-13 14:57:45,869 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] FINISH, UploadStreamVDSCommand, log id: 7ea3b9bd >2014-08-13 14:57:45,871 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [61868b0f] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 6c55a9e5-c4ec-47fb-a618-d6af09526434 >2014-08-13 14:57:45,871 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-41) [61868b0f] CommandMultiAsyncTasks::AttachTask: Attaching task c0c0e61e-3897-4fd7-a504-b9593a0f952e to command 6c55a9e5-c4ec-47fb-a618-d6af09526434. >2014-08-13 14:57:46,060 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-41) [61868b0f] Adding task c0c0e61e-3897-4fd7-a504-b9593a0f952e (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 14:57:46,152 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 14:57:46,154 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4c16b586-1b20-4f3a-9001-43521e3bb552, imageId = 655fad6a-3526-407e-a7d9-2d5e09102565), log id: 17275434 >2014-08-13 14:57:46,155 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 14:57:46,155 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4c16b586-1b20-4f3a-9001-43521e3bb552 > volUUID=655fad6a-3526-407e-a7d9-2d5e09102565 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Wed Aug 13 14:57:40 IDT 2014","Size":61440} > >2014-08-13 14:57:46,574 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] FINISH, SetVolumeDescriptionVDSCommand, log id: 17275434 >2014-08-13 14:57:46,620 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4cab3024-3f6d-435a-bf22-a6e7daac3bd9, imageId = 612a0f89-926c-48c5-9804-b258314e4838), log id: 47195f17 >2014-08-13 14:57:46,621 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 14:57:46,621 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4cab3024-3f6d-435a-bf22-a6e7daac3bd9 > volUUID=612a0f89-926c-48c5-9804-b258314e4838 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Wed Aug 13 12:55:54 IDT 2014"} > >2014-08-13 14:57:47,038 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [61868b0f] FINISH, SetVolumeDescriptionVDSCommand, log id: 47195f17 >2014-08-13 14:57:47,170 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-41) [76f90d5] Running command: UploadStreamCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-13 14:57:47,172 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [76f90d5] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 6d3918d5 >2014-08-13 14:57:47,172 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [76f90d5] -- executeVdsBrokerCommand: >2014-08-13 14:57:47,172 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [76f90d5] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGUID=4cab3024-3f6d-435a-bf22-a6e7daac3bd9 > volUUID=612a0f89-926c-48c5-9804-b258314e4838 > size=61,440 > >2014-08-13 14:57:49,130 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [76f90d5] FINISH, UploadStreamVDSCommand, log id: 6d3918d5 >2014-08-13 14:57:49,131 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-41) [76f90d5] CommandMultiAsyncTasks::AttachTask: Attaching task a9736939-f8ee-4e9c-96fd-46c3df9b2b73 to command 6c55a9e5-c4ec-47fb-a618-d6af09526434. >2014-08-13 14:57:49,182 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-41) [76f90d5] Adding task a9736939-f8ee-4e9c-96fd-46c3df9b2b73 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 14:57:49,306 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-41) [76f90d5] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-13 14:57:49,307 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-41) [76f90d5] Trying to release exclusive lock which does not exist, lock key: d54e6e6c-81f2-41f2-90ab-7e3c2457938aSTORAGE >2014-08-13 14:57:49,307 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-41) [76f90d5] Trying to release exclusive lock which does not exist, lock key: 4c16b586-1b20-4f3a-9001-43521e3bb552DISK >2014-08-13 14:57:49,307 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-41) [76f90d5] Trying to release exclusive lock which does not exist, lock key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9DISK >2014-08-13 14:57:49,308 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-41) [76f90d5] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 14:57:49,310 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [76f90d5] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4cab3024-3f6d-435a-bf22-a6e7daac3bd9, imageId = 612a0f89-926c-48c5-9804-b258314e4838), log id: 46d9d8b9 >2014-08-13 14:57:49,311 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [76f90d5] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 14:57:49,311 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [76f90d5] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4cab3024-3f6d-435a-bf22-a6e7daac3bd9 > volUUID=612a0f89-926c-48c5-9804-b258314e4838 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Wed Aug 13 14:57:40 IDT 2014","Size":61440} > >2014-08-13 14:57:49,678 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [76f90d5] FINISH, SetVolumeDescriptionVDSCommand, log id: 46d9d8b9 >2014-08-13 14:57:49,788 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [76f90d5] BaseAsyncTask::startPollingTask: Starting to poll task c0c0e61e-3897-4fd7-a504-b9593a0f952e. >2014-08-13 14:57:49,788 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [76f90d5] BaseAsyncTask::startPollingTask: Starting to poll task a9736939-f8ee-4e9c-96fd-46c3df9b2b73. >2014-08-13 14:57:49,830 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-41) [20747331] Lock Acquired to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 14:57:49,832 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-41) [20747331] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-13 14:57:49,840 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [20747331] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 458d431f >2014-08-13 14:57:49,841 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [20747331] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 14:57:49,841 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [20747331] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Wed Aug 13 12:56:03 IDT 2014"} > >2014-08-13 14:57:50,175 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [20747331] FINISH, SetVolumeDescriptionVDSCommand, log id: 458d431f >2014-08-13 14:57:50,349 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-41) [f796586] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-13 14:57:50,351 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [f796586] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 6247bc12 >2014-08-13 14:57:50,351 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [f796586] -- executeVdsBrokerCommand: >2014-08-13 14:57:50,352 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [f796586] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > size=81,920 > >2014-08-13 14:57:51,136 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-8) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-13 14:57:51,520 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-8) SPMAsyncTask::PollTask: Polling task c0c0e61e-3897-4fd7-a504-b9593a0f952e (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 14:57:51,520 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-8) BaseAsyncTask::onTaskEndSuccess: Task c0c0e61e-3897-4fd7-a504-b9593a0f952e (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 14:57:51,521 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-8) Task with DB Task ID fa22537e-e864-4a89-9e99-68790c4b7b59 and VDSM Task ID a9736939-f8ee-4e9c-96fd-46c3df9b2b73 is in state Polling. End action for command 6c55a9e5-c4ec-47fb-a618-d6af09526434 will proceed when all the entitys tasks are completed. >2014-08-13 14:57:51,521 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-8) SPMAsyncTask::PollTask: Polling task a9736939-f8ee-4e9c-96fd-46c3df9b2b73 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 14:57:51,522 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-8) BaseAsyncTask::onTaskEndSuccess: Task a9736939-f8ee-4e9c-96fd-46c3df9b2b73 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 14:57:51,522 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-8) CommandAsyncTask::endActionIfNecessary: All tasks of command 6c55a9e5-c4ec-47fb-a618-d6af09526434 has ended -> executing endAction >2014-08-13 14:57:51,522 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-8) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 6c55a9e5-c4ec-47fb-a618-d6af09526434): calling endAction . >2014-08-13 14:57:51,523 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-08-13 14:57:51,539 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-14) [519f6d28] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-08-13 14:57:51,540 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-14) [519f6d28] Trying to release exclusive lock which does not exist, lock key: d54e6e6c-81f2-41f2-90ab-7e3c2457938aSTORAGE >2014-08-13 14:57:51,540 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-14) [519f6d28] Trying to release exclusive lock which does not exist, lock key: 4c16b586-1b20-4f3a-9001-43521e3bb552DISK >2014-08-13 14:57:51,541 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-14) [519f6d28] Trying to release exclusive lock which does not exist, lock key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9DISK >2014-08-13 14:57:51,541 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-14) [519f6d28] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 14:57:51,542 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) [519f6d28] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-08-13 14:57:51,543 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) [519f6d28] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-08-13 14:57:51,543 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) [519f6d28] SPMAsyncTask::ClearAsyncTask: Attempting to clear task a9736939-f8ee-4e9c-96fd-46c3df9b2b73 >2014-08-13 14:57:51,544 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) [519f6d28] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = a9736939-f8ee-4e9c-96fd-46c3df9b2b73), log id: 30cba99a >2014-08-13 14:57:51,548 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) [519f6d28] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=a9736939-f8ee-4e9c-96fd-46c3df9b2b73), log id: aebd67d >2014-08-13 14:57:51,922 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) [519f6d28] FINISH, HSMClearTaskVDSCommand, log id: aebd67d >2014-08-13 14:57:51,922 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) [519f6d28] FINISH, SPMClearTaskVDSCommand, log id: 30cba99a >2014-08-13 14:57:51,947 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) [519f6d28] BaseAsyncTask::removeTaskFromDB: Removed task a9736939-f8ee-4e9c-96fd-46c3df9b2b73 from DataBase >2014-08-13 14:57:51,948 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) [519f6d28] SPMAsyncTask::ClearAsyncTask: Attempting to clear task c0c0e61e-3897-4fd7-a504-b9593a0f952e >2014-08-13 14:57:51,948 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) [519f6d28] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = c0c0e61e-3897-4fd7-a504-b9593a0f952e), log id: 448a54ea >2014-08-13 14:57:51,950 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) [519f6d28] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=c0c0e61e-3897-4fd7-a504-b9593a0f952e), log id: 55ccf1cd >2014-08-13 14:57:52,240 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) [519f6d28] FINISH, HSMClearTaskVDSCommand, log id: 55ccf1cd >2014-08-13 14:57:52,240 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-14) [519f6d28] FINISH, SPMClearTaskVDSCommand, log id: 448a54ea >2014-08-13 14:57:52,278 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-14) [519f6d28] BaseAsyncTask::removeTaskFromDB: Removed task c0c0e61e-3897-4fd7-a504-b9593a0f952e from DataBase >2014-08-13 14:57:52,279 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-14) [519f6d28] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 6c55a9e5-c4ec-47fb-a618-d6af09526434 >2014-08-13 14:57:53,133 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [f796586] FINISH, UploadStreamVDSCommand, log id: 6247bc12 >2014-08-13 14:57:53,135 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-41) [f796586] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command b554744b-3e17-460e-b7ca-01052d8cc4ea >2014-08-13 14:57:53,135 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-41) [f796586] CommandMultiAsyncTasks::AttachTask: Attaching task 687416b7-a521-49a3-ace9-7e01314c632b to command b554744b-3e17-460e-b7ca-01052d8cc4ea. >2014-08-13 14:57:53,164 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-41) [f796586] Adding task 687416b7-a521-49a3-ace9-7e01314c632b (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 14:57:53,255 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-41) [f796586] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-13 14:57:53,256 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-41) [f796586] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 14:57:53,260 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [f796586] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 597425e7 >2014-08-13 14:57:53,260 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [f796586] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 14:57:53,261 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [f796586] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Wed Aug 13 14:57:49 IDT 2014","Size":81920} > >2014-08-13 14:57:54,146 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [f796586] FINISH, SetVolumeDescriptionVDSCommand, log id: 597425e7 >2014-08-13 14:57:54,168 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [f796586] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 588af502 >2014-08-13 14:57:54,580 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [f796586] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 14:57:54,580 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [f796586] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Wed Aug 13 12:56:03 IDT 2014"} > >2014-08-13 14:57:54,869 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [f796586] FINISH, SetVolumeDescriptionVDSCommand, log id: 588af502 >2014-08-13 14:57:54,994 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-13 14:57:54,996 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 693dbb44 >2014-08-13 14:57:54,996 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] -- executeVdsBrokerCommand: >2014-08-13 14:57:54,996 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > size=81,920 > >2014-08-13 14:57:58,058 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] FINISH, UploadStreamVDSCommand, log id: 693dbb44 >2014-08-13 14:57:58,060 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] CommandMultiAsyncTasks::AttachTask: Attaching task eba1124a-492a-4c6f-8436-a3c789e27f18 to command b554744b-3e17-460e-b7ca-01052d8cc4ea. >2014-08-13 14:57:58,107 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] Adding task eba1124a-492a-4c6f-8436-a3c789e27f18 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-13 14:57:58,165 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-13 14:57:58,166 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-13 14:57:58,166 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-13 14:57:58,167 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-13 14:57:58,167 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 14:57:58,171 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 2cd1c519 >2014-08-13 14:57:58,173 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-13 14:57:58,173 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Wed Aug 13 14:57:49 IDT 2014","Size":81920} > >2014-08-13 14:57:58,572 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] FINISH, SetVolumeDescriptionVDSCommand, log id: 2cd1c519 >2014-08-13 14:57:58,604 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] BaseAsyncTask::startPollingTask: Starting to poll task 687416b7-a521-49a3-ace9-7e01314c632b. >2014-08-13 14:57:58,604 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) [36f8cdc6] BaseAsyncTask::startPollingTask: Starting to poll task eba1124a-492a-4c6f-8436-a3c789e27f18. >2014-08-13 14:58:01,523 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-56) Polling and updating Async Tasks: 4 tasks, 2 tasks to poll now >2014-08-13 14:58:02,240 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-56) SPMAsyncTask::PollTask: Polling task 687416b7-a521-49a3-ace9-7e01314c632b (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 14:58:02,241 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-56) BaseAsyncTask::onTaskEndSuccess: Task 687416b7-a521-49a3-ace9-7e01314c632b (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 14:58:02,242 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-56) Task with DB Task ID e252a827-766d-46dc-a5b6-77fe3433f102 and VDSM Task ID eba1124a-492a-4c6f-8436-a3c789e27f18 is in state Polling. End action for command b554744b-3e17-460e-b7ca-01052d8cc4ea will proceed when all the entitys tasks are completed. >2014-08-13 14:58:02,243 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-56) SPMAsyncTask::PollTask: Polling task eba1124a-492a-4c6f-8436-a3c789e27f18 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-13 14:58:02,244 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-56) BaseAsyncTask::onTaskEndSuccess: Task eba1124a-492a-4c6f-8436-a3c789e27f18 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-13 14:58:02,245 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-56) CommandAsyncTask::endActionIfNecessary: All tasks of command b554744b-3e17-460e-b7ca-01052d8cc4ea has ended -> executing endAction >2014-08-13 14:58:02,245 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-56) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: b554744b-3e17-460e-b7ca-01052d8cc4ea): calling endAction . >2014-08-13 14:58:02,246 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-08-13 14:58:02,271 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-34) [20747331] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-08-13 14:58:02,272 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-34) [20747331] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-13 14:58:02,272 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-34) [20747331] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-13 14:58:02,273 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-34) [20747331] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-13 14:58:02,273 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-34) [20747331] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-13 14:58:02,274 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-34) [20747331] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-13 14:58:02,275 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) [20747331] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-08-13 14:58:02,276 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) [20747331] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-08-13 14:58:02,276 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) [20747331] SPMAsyncTask::ClearAsyncTask: Attempting to clear task eba1124a-492a-4c6f-8436-a3c789e27f18 >2014-08-13 14:58:02,277 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [20747331] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = eba1124a-492a-4c6f-8436-a3c789e27f18), log id: 31175c75 >2014-08-13 14:58:02,288 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [20747331] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=eba1124a-492a-4c6f-8436-a3c789e27f18), log id: 74b71e80 >2014-08-13 14:58:02,697 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [20747331] FINISH, HSMClearTaskVDSCommand, log id: 74b71e80 >2014-08-13 14:58:02,698 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [20747331] FINISH, SPMClearTaskVDSCommand, log id: 31175c75 >2014-08-13 14:58:02,727 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) [20747331] BaseAsyncTask::removeTaskFromDB: Removed task eba1124a-492a-4c6f-8436-a3c789e27f18 from DataBase >2014-08-13 14:58:02,727 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) [20747331] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 687416b7-a521-49a3-ace9-7e01314c632b >2014-08-13 14:58:02,728 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [20747331] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 687416b7-a521-49a3-ace9-7e01314c632b), log id: 5dd3b678 >2014-08-13 14:58:02,729 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [20747331] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=687416b7-a521-49a3-ace9-7e01314c632b), log id: 7670b067 >2014-08-13 14:58:03,061 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [20747331] FINISH, HSMClearTaskVDSCommand, log id: 7670b067 >2014-08-13 14:58:03,061 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-34) [20747331] FINISH, SPMClearTaskVDSCommand, log id: 5dd3b678 >2014-08-13 14:58:03,110 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-34) [20747331] BaseAsyncTask::removeTaskFromDB: Removed task 687416b7-a521-49a3-ace9-7e01314c632b from DataBase >2014-08-13 14:58:03,111 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-34) [20747331] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity b554744b-3e17-460e-b7ca-01052d8cc4ea >2014-08-13 14:58:59,941 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-45) Setting new tasks map. The map contains now 2 tasks >2014-08-13 14:59:29,943 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-50) Setting new tasks map. The map contains now 0 tasks >2014-08-13 14:59:29,944 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-50) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-13 15:56:24,135 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-44) Principal admin::internal synchronized >2014-08-13 15:57:58,605 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-77) Attempting to update VMs/Templates Ovf. >2014-08-13 16:56:24,285 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-60) Principal admin::internal synchronized >2014-08-13 16:57:58,636 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-93) Attempting to update VMs/Templates Ovf. >2014-08-18 09:42:48,938 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-18 09:42:49,065 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-18 09:42:49,066 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-18 09:42:49,068 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-18 09:42:49,070 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-18 09:42:49,071 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-18 09:42:49,072 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-18 09:42:49,073 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-18 09:42:49,073 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-18 09:42:49,074 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-18 09:42:49,075 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-18 09:42:49,075 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-18 09:42:49,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-18 09:42:49,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-18 09:42:49,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-18 09:42:49,078 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-18 09:42:49,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-18 09:42:49,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-18 09:42:49,079 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-18 09:42:49,080 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-18 09:42:49,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-18 09:42:49,081 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-18 09:42:49,083 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DB_USER" is "engine". >2014-08-18 09:42:49,083 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-18 09:42:49,084 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-18 09:42:49,085 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-18 09:42:49,085 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-18 09:42:49,086 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-18 09:42:49,087 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-18 09:42:49,088 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-18 09:42:49,088 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-18 09:42:49,089 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-18 09:42:49,090 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-18 09:42:49,090 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-18 09:42:49,091 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-18 09:42:49,091 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-18 09:42:49,092 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-18 09:42:49,093 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-18 09:42:49,094 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-18 09:42:49,094 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-18 09:42:49,095 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-18 09:42:49,096 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-18 09:42:49,096 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-18 09:42:49,097 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-18 09:42:49,098 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-18 09:42:49,099 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-18 09:42:49,099 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-18 09:42:49,100 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-18 09:42:49,101 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-18 09:42:49,101 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-18 09:42:49,102 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-18 09:42:49,102 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-18 09:42:49,103 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-18 09:42:49,103 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-18 09:42:49,104 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-18 09:42:49,104 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-18 09:42:49,105 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-18 09:42:49,105 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-18 09:42:49,106 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-18 09:42:49,106 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-18 09:42:49,107 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-18 09:42:49,107 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-18 09:42:49,111 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-18 09:42:49,111 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_USER" is "ovirt". >2014-08-18 09:42:49,112 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-18 09:42:49,112 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-18 09:42:49,113 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-18 09:42:49,115 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-18 09:42:49,118 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-18 09:42:49,118 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-7) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-18 09:42:49,519 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Start initializing Backend >2014-08-18 09:42:49,717 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 09:42:49,802 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 09:42:49,843 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-18 09:42:49,864 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-1) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-18 09:42:50,078 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Running ovirt-engine 3.5.0.0 >2014-08-18 09:42:50,078 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Start initializing dictionaries >2014-08-18 09:42:50,080 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Finished initializing dictionaries >2014-08-18 09:42:50,081 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing AuditLogCleanupManager >2014-08-18 09:42:50,082 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-18 09:42:50,100 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing AuditLogCleanupManager >2014-08-18 09:42:50,101 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing CommandEntityCleanupManager >2014-08-18 09:42:50,101 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-18 09:42:50,102 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing CommandEntityCleanupManager >2014-08-18 09:42:50,104 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Start initializing TagsDirector >2014-08-18 09:42:50,113 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Tag root added to tree >2014-08-18 09:42:50,162 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Finished initializing TagsDirector >2014-08-18 09:42:50,163 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Start initializing IsoDomainListSyncronizer >2014-08-18 09:42:50,167 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Finished initializing IsoDomainListSyncronizer >2014-08-18 09:42:50,253 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Completed initializing handlers >2014-08-18 09:42:50,273 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-08-18 09:42:50,285 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-18 09:42:50,286 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-08-18 09:42:50,287 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-08-18 09:42:50,288 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-08-18 09:42:50,288 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Mark incomplete jobs as UNKNOWN >2014-08-18 09:42:50,325 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Start initializing JobRepositoryCleanupManager >2014-08-18 09:42:50,326 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Finished initializing JobRepositoryCleanupManager >2014-08-18 09:42:50,327 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Start initializing AutoRecoveryManager >2014-08-18 09:42:50,328 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Finished initializing AutoRecoveryManager >2014-08-18 09:42:50,330 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Start initializing ExecutionMessageDirector >2014-08-18 09:42:50,332 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Finished initializing ExecutionMessageDirector >2014-08-18 09:42:50,382 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Start of enabled extensions list >2014-08-18 09:42:50,382 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-18 09:42:50,383 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-18 09:42:50,384 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) End of enabled extensions list >2014-08-18 09:42:50,475 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-4) Start initializing DbUserCacheManager >2014-08-18 09:42:50,476 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-4) Finished initializing DbUserCacheManager >2014-08-18 09:42:50,486 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-4) Initialization of AsyncTaskManager completed successfully. >2014-08-18 09:42:50,489 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-4) Start initializing ResourceManager >2014-08-18 09:42:50,503 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-18 09:42:50,567 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-4) Entered VdsManager constructor >2014-08-18 09:42:50,593 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-4) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-18 09:42:50,683 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-4) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-18 09:42:50,684 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-4) Entered VdsManager constructor >2014-08-18 09:42:50,696 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-4) Initialize vdsBroker (galactica,54,321) >2014-08-18 09:42:50,723 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-4) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-18 09:42:50,744 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-4) Finished initializing ResourceManager >2014-08-18 09:42:50,748 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-4) Initialization of OvfDataUpdater completed successfully. >2014-08-18 09:42:50,748 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Start scheduling to enable vds load balancer >2014-08-18 09:42:50,749 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Finished scheduling to enable vds load balancer >2014-08-18 09:42:50,749 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Start HA Reservation check >2014-08-18 09:42:50,750 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Finished HA Reservation check >2014-08-18 09:42:50,757 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-4) Init VM custom properties utilities >2014-08-18 09:42:50,758 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-4) Init device custom properties utilities >2014-08-18 09:42:50,762 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Initializing Scheduling manager >2014-08-18 09:42:50,789 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-18 09:42:50,790 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-18 09:42:50,791 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-18 09:42:50,813 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) External scheduler disabled, discovery skipped >2014-08-18 09:42:50,816 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-4) Initialized Scheduling manager >2014-08-18 09:42:50,817 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-4) Initializing DWH Heart Beat >2014-08-18 09:42:50,818 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-4) DWH Heart Beat initialized >2014-08-18 09:42:53,756 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: h2 >2014-08-18 09:42:54,508 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 51dd6085 >2014-08-18 09:42:54,520 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 51dd6085 >2014-08-18 09:42:54,530 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-18 09:42:54,537 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: UNASSIGNED not exist in string table >2014-08-18 09:42:54,538 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-18 09:42:54,538 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-18 09:42:54,539 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-18 09:42:54,540 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-18 09:42:54,541 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-18 09:42:54,542 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-18 09:42:54,543 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-18 09:42:54,544 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-18 09:42:54,545 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-18 09:42:54,546 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-18 09:42:54,546 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STOPPED not exist in string table >2014-08-18 09:42:54,547 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STARTED not exist in string table >2014-08-18 09:42:54,548 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_ERROR not exist in string table >2014-08-18 09:42:54,548 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-08-18 09:42:54,549 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-08-18 09:42:54,549 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-08-18 09:42:54,550 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-08-18 09:42:54,551 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-08-18 09:42:54,551 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-08-18 09:42:54,552 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-18 09:42:54,552 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-18 09:42:54,553 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-18 09:42:54,554 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-18 09:42:54,754 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-18 09:42:54,754 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is running with disabled SELinux. >2014-08-18 09:42:55,345 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h1 >2014-08-18 09:42:55,492 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [62738171] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-18 09:43:00,934 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-23) [18256a44] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-18 09:43:00,948 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [18256a44] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-18 09:43:00,954 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [18256a44] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-18 09:43:01,039 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [18256a44] Correlation ID: 18256a44, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-18 09:43:01,140 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) [18256a44] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-18 09:43:01,147 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) [18256a44] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-18 09:43:01,172 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [18256a44] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-18 09:43:01,190 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [18256a44] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 4f4fe9c8 >2014-08-18 09:43:01,246 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [18256a44] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 30 GB of free space >2014-08-18 09:43:01,271 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [18256a44] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 30 GB of free space >2014-08-18 09:43:01,272 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [18256a44] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-18 09:43:01,279 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [18256a44] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 3694a19f >2014-08-18 09:43:01,286 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [18256a44] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 3694a19f >2014-08-18 09:43:01,286 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [18256a44] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 4f4fe9c8 >2014-08-18 09:43:01,287 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [18256a44] Discovered no tasks on Storage Pool Default >2014-08-18 09:44:20,038 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-4) Running command: LoginAdminUserCommand internal: false. >2014-08-18 09:44:20,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-18 09:44:23,772 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-5) Running command: LoginUserCommand internal: false. >2014-08-18 09:48:57,798 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [5cf14457] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-18 09:48:58,006 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-43) [5cf14457] Running command: HibernateVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-18 09:48:58,042 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-43) [5cf14457] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@db70e563), log id: 5399ddfa >2014-08-18 09:48:58,045 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-43) [5cf14457] FINISH, UpdateVmDynamicDataVDSCommand, log id: 5399ddfa >2014-08-18 09:48:58,059 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-43) [5cf14457] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 2ce3c107-135a-463a-834b-89bdbc8458aa, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = f05665c3-f3e6-4555-b1e1-6db2c50da904, newImageDescription = ), log id: 7016a7ae >2014-08-18 09:48:58,064 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-43) [5cf14457] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-18 09:48:58,912 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-43) [5cf14457] FINISH, CreateImageVDSCommand, return: f05665c3-f3e6-4555-b1e1-6db2c50da904, log id: 7016a7ae >2014-08-18 09:48:58,969 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-43) [5cf14457] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 8af18b2f-0205-40ae-ad05-c87408aba7a2 >2014-08-18 09:48:58,970 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-43) [5cf14457] CommandMultiAsyncTasks::AttachTask: Attaching task 3273d6ec-9bf8-4a89-8969-e309bee44fc0 to command 8af18b2f-0205-40ae-ad05-c87408aba7a2. >2014-08-18 09:48:58,995 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-43) [5cf14457] Adding task 3273d6ec-9bf8-4a89-8969-e309bee44fc0 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-18 09:48:59,017 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-43) [5cf14457] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 463eca98-08e5-47be-8be1-e620e22cc872, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = e1a312ce-2c10-4e4f-a3bf-3dd70b7543e5, newImageDescription = ), log id: 65129739 >2014-08-18 09:48:59,207 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-43) [5cf14457] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-18 09:48:59,759 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-43) [5cf14457] FINISH, CreateImageVDSCommand, return: e1a312ce-2c10-4e4f-a3bf-3dd70b7543e5, log id: 65129739 >2014-08-18 09:48:59,795 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-43) [5cf14457] CommandMultiAsyncTasks::AttachTask: Attaching task 7875d36c-63c9-4484-a7da-b861522e371e to command 8af18b2f-0205-40ae-ad05-c87408aba7a2. >2014-08-18 09:48:59,811 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-43) [5cf14457] Adding task 7875d36c-63c9-4484-a7da-b861522e371e (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-18 09:48:59,820 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-43) [5cf14457] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@339750da), log id: 1caf8242 >2014-08-18 09:48:59,827 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-43) [5cf14457] FINISH, UpdateVmDynamicDataVDSCommand, log id: 1caf8242 >2014-08-18 09:48:59,877 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-43) [5cf14457] Correlation ID: 5cf14457, Job ID: 8f427e9e-ddb2-43f6-99e5-a52bbc1ddccb, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm3 was initiated by User admin (Host: h1). >2014-08-18 09:48:59,878 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-43) [5cf14457] BaseAsyncTask::startPollingTask: Starting to poll task 3273d6ec-9bf8-4a89-8969-e309bee44fc0. >2014-08-18 09:48:59,878 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-43) [5cf14457] BaseAsyncTask::startPollingTask: Starting to poll task 7875d36c-63c9-4484-a7da-b861522e371e. >2014-08-18 09:48:59,879 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-43) [5cf14457] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-18 09:49:00,485 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-24) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-18 09:49:00,503 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-24) SPMAsyncTask::PollTask: Polling task 7875d36c-63c9-4484-a7da-b861522e371e (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-18 09:49:00,503 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-24) SPMAsyncTask::PollTask: Polling task 3273d6ec-9bf8-4a89-8969-e309bee44fc0 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-18 09:49:00,504 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-24) Finished polling Tasks, will poll again in 10 seconds. >2014-08-18 09:49:03,264 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from PreparingForHibernate --> Up >2014-08-18 09:49:10,512 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) SPMAsyncTask::PollTask: Polling task 7875d36c-63c9-4484-a7da-b861522e371e (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-18 09:49:10,538 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) BaseAsyncTask::onTaskEndSuccess: Task 7875d36c-63c9-4484-a7da-b861522e371e (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-18 09:49:10,538 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-49) Task with DB Task ID c5773d0f-0a16-4607-a033-f793c914237e and VDSM Task ID 3273d6ec-9bf8-4a89-8969-e309bee44fc0 is in state Polling. End action for command 8af18b2f-0205-40ae-ad05-c87408aba7a2 will proceed when all the entitys tasks are completed. >2014-08-18 09:49:10,540 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) SPMAsyncTask::PollTask: Polling task 3273d6ec-9bf8-4a89-8969-e309bee44fc0 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-18 09:49:10,554 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) BaseAsyncTask::onTaskEndSuccess: Task 3273d6ec-9bf8-4a89-8969-e309bee44fc0 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-18 09:49:10,554 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-49) CommandAsyncTask::endActionIfNecessary: All tasks of command 8af18b2f-0205-40ae-ad05-c87408aba7a2 has ended -> executing endAction >2014-08-18 09:49:10,555 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-49) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 8af18b2f-0205-40ae-ad05-c87408aba7a2): calling endAction . >2014-08-18 09:49:10,556 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-9) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-18 09:49:10,604 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-9) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-18 09:49:10,606 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-9) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,2ce3c107-135a-463a-834b-89bdbc8458aa,f05665c3-f3e6-4555-b1e1-6db2c50da904,463eca98-08e5-47be-8be1-e620e22cc872,e1a312ce-2c10-4e4f-a3bf-3dd70b7543e5), log id: 1ae84667 >2014-08-18 09:49:10,609 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-9) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,2ce3c107-135a-463a-834b-89bdbc8458aa,f05665c3-f3e6-4555-b1e1-6db2c50da904,463eca98-08e5-47be-8be1-e620e22cc872,e1a312ce-2c10-4e4f-a3bf-3dd70b7543e5), log id: 4d1934f0 >2014-08-18 09:49:10,616 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-9) FINISH, HibernateBrokerVDSCommand, log id: 4d1934f0 >2014-08-18 09:49:10,628 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-9) FINISH, HibernateVDSCommand, log id: 1ae84667 >2014-08-18 09:49:10,630 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-9) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-18 09:49:10,630 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-9) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-18 09:49:10,645 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-9) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 7875d36c-63c9-4484-a7da-b861522e371e >2014-08-18 09:49:10,646 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-9) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 7875d36c-63c9-4484-a7da-b861522e371e), log id: 7e145cb5 >2014-08-18 09:49:10,648 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-9) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=7875d36c-63c9-4484-a7da-b861522e371e), log id: 6a2cf6d2 >2014-08-18 09:49:10,782 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-9) FINISH, HSMClearTaskVDSCommand, log id: 6a2cf6d2 >2014-08-18 09:49:10,782 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-9) FINISH, SPMClearTaskVDSCommand, log id: 7e145cb5 >2014-08-18 09:49:10,794 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-9) BaseAsyncTask::removeTaskFromDB: Removed task 7875d36c-63c9-4484-a7da-b861522e371e from DataBase >2014-08-18 09:49:10,794 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-9) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 3273d6ec-9bf8-4a89-8969-e309bee44fc0 >2014-08-18 09:49:10,795 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-9) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 3273d6ec-9bf8-4a89-8969-e309bee44fc0), log id: 77001e17 >2014-08-18 09:49:10,796 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-9) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=3273d6ec-9bf8-4a89-8969-e309bee44fc0), log id: 28b7b50a >2014-08-18 09:49:10,920 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-9) FINISH, HSMClearTaskVDSCommand, log id: 28b7b50a >2014-08-18 09:49:10,921 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-9) FINISH, SPMClearTaskVDSCommand, log id: 77001e17 >2014-08-18 09:49:10,935 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-9) BaseAsyncTask::removeTaskFromDB: Removed task 3273d6ec-9bf8-4a89-8969-e309bee44fc0 from DataBase >2014-08-18 09:49:10,935 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-9) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 8af18b2f-0205-40ae-ad05-c87408aba7a2 >2014-08-18 09:49:15,526 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-62) START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, force=false, secondsToWait=0, gracefully=false, reason=), log id: 30e6347d >2014-08-18 09:49:15,535 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, DestroyVDSCommand, log id: 30e6347d >2014-08-18 09:49:15,590 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm3 on Host h1 is suspended. >2014-08-18 09:49:15,592 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) VM vm3 (5b1ee280-4251-489a-899c-8e9dac73c327) is running in db and not running in VDS h1 >2014-08-18 09:49:15,593 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-62) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: 4cbfdd48 >2014-08-18 09:49:15,598 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-62) FINISH, FullListVdsCommand, return: [], log id: 4cbfdd48 >2014-08-18 09:49:15,651 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-18) [77c51ac7] Running command: ProcessDownVmCommand internal: true. >2014-08-18 09:49:38,836 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-4) [206012b0] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-18 09:49:38,933 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [206012b0] START, IsVmDuringInitiatingVDSCommand( vmId = 5b1ee280-4251-489a-899c-8e9dac73c327), log id: 12d6e056 >2014-08-18 09:49:38,933 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [206012b0] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 12d6e056 >2014-08-18 09:49:39,073 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-44) [206012b0] Running command: RunVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-18 09:49:39,197 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-44) [206012b0] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = ISCSI, connectionList = [{ id: bd327084-78e4-4b84-9a74-9cea2d43c971, connection: 10.35.16.25, iqn: iqn.2011-05.com.redhat:achub-iscsi, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 135b5b37 >2014-08-18 09:49:40,768 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-44) [206012b0] FINISH, ConnectStorageServerVDSCommand, return: {bd327084-78e4-4b84-9a74-9cea2d43c971=0}, log id: 135b5b37 >2014-08-18 09:49:40,776 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-44) [206012b0] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@f4dd653d), log id: 6656e06 >2014-08-18 09:49:40,787 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-44) [206012b0] FINISH, UpdateVmDynamicDataVDSCommand, log id: 6656e06 >2014-08-18 09:49:40,806 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-44) [206012b0] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, vm=VM [vm3]), log id: 3a68ca65 >2014-08-18 09:49:40,822 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-44) [206012b0] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, vm=VM [vm3]), log id: 4e59bc4a >2014-08-18 09:49:40,936 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-44) [206012b0] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,hiberVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,2ce3c107-135a-463a-834b-89bdbc8458aa,f05665c3-f3e6-4555-b1e1-6db2c50da904,463eca98-08e5-47be-8be1-e620e22cc872,e1a312ce-2c10-4e4f-a3bf-3dd70b7543e5,transparentHugePages=true,vmId=5b1ee280-4251-489a-899c-8e9dac73c327,acpiEnable=true,custom={device_fd7fb931-5e78-43c1-8d65-a3c078938c85device_f125349d-3048-48f9-9c33-08983de67abcdevice_b6835a98-76cb-4b8c-b0b1-6bf1b0424b47=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=b6835a98-76cb-4b8c-b0b1-6bf1b0424b47, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_fd7fb931-5e78-43c1-8d65-a3c078938c85device_f125349d-3048-48f9-9c33-08983de67abc=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=f125349d-3048-48f9-9c33-08983de67abc, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_fd7fb931-5e78-43c1-8d65-a3c078938c85=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=fd7fb931-5e78-43c1-8d65-a3c078938c85, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_fd7fb931-5e78-43c1-8d65-a3c078938c85device_f125349d-3048-48f9-9c33-08983de67abcdevice_b6835a98-76cb-4b8c-b0b1-6bf1b0424b47device_75b554af-5385-4f20-a637-49ea92e3c760=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=75b554af-5385-4f20-a637-49ea92e3c760, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_fd7fb931-5e78-43c1-8d65-a3c078938c85device_f125349d-3048-48f9-9c33-08983de67abcdevice_b6835a98-76cb-4b8c-b0b1-6bf1b0424b47device_75b554af-5385-4f20-a637-49ea92e3c760device_96c8ccf7-eae6-4354-ac32-442f636eee8c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=f09597d8-8cc8-4549-863f-b96975014b01}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=f76be01f-4e52-4d67-8e1b-0856337e9929}, {shared=false, index=0, volumeID=26308abd-8540-410e-8018-58cd2157084d, propagateErrors=off, format=cow, type=disk, iface=virtio, bootOrder=1, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, imageID=b5576e45-7f13-465f-b26f-05d52faf28d1, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=b5576e45-7f13-465f-b26f-05d52faf28d1}, {shared=false, iface=virtio, GUID=3600144f09dbd050000004e1994c60005, address={bus=0x00, domain=0x0000, slot=0x07, type=pci, function=0x0}, specParams={}, optional=false, propagateErrors=off, device=lun, format=raw, type=disk, readonly=false, deviceId=bdca3776-6597-4323-84f1-4ea75f7d269d}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:53, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=68e93502-4ad4-42a0-9a23-dfa3f67090f5}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=403887de-d0b8-4108-8b43-b4b5d5413859}, {address={bus=0x00, domain=0x0000, slot=0x05, type=pci, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c}],vmName=vm3,cpuType=SandyBridge,fileTransferEnable=true >2014-08-18 09:49:41,002 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-44) [206012b0] FINISH, CreateVDSCommand, log id: 4e59bc4a >2014-08-18 09:49:41,052 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-44) [206012b0] FINISH, CreateVmVDSCommand, return: RestoringState, log id: 3a68ca65 >2014-08-18 09:49:41,053 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-44) [206012b0] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-18 09:49:41,069 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-44) [206012b0] Correlation ID: 206012b0, Job ID: ddee0c19-b0ef-4877-8293-0ed6425cc820, Call Stack: null, Custom Event ID: -1, Message: VM vm3 was started by admin (Host: h1). >2014-08-18 09:49:43,716 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from RestoringState --> Up >2014-08-18 09:49:43,718 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-39) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: 37586b3b >2014-08-18 09:49:43,734 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-39) FINISH, FullListVdsCommand, return: [{displaySecurePort=5900, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, acpiEnable=true, custom={device_a0176251-79b2-48df-ba52-49dc7fd73498=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=a0176251-79b2-48df-ba52-49dc7fd73498, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48device_611f9766-6112-4e87-86c1-42cc5f30a38c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=611f9766-6112-4e87-86c1-42cc5f30a38c, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48device_611f9766-6112-4e87-86c1-42cc5f30a38cdevice_96c8ccf7-eae6-4354-ac32-442f636eee8c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=2d383e02-c4b2-4b55-b54c-60d73acbce48, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=11829, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@69c321eb, vmName=vm3, fileTransferEnable=true, cpuType=SandyBridge}], log id: 37586b3b >2014-08-18 09:49:43,746 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Received a memballoon Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-18 09:49:43,747 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Received a spice Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5900, type=graphics} >2014-08-18 09:49:43,877 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-39) Correlation ID: 206012b0, Job ID: ddee0c19-b0ef-4877-8293-0ed6425cc820, Call Stack: null, Custom Event ID: -1, Message: VM vm3 started on Host h1 >2014-08-18 09:49:43,889 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-18 09:49:43,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 2ce3c107-135a-463a-834b-89bdbc8458aa, postZeros = false, forceDelete = false), log id: 53e0f151 >2014-08-18 09:49:44,371 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] FINISH, DeleteImageGroupVDSCommand, log id: 53e0f151 >2014-08-18 09:49:44,501 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 9201720a-07a3-4feb-a837-fa092b7b02cc >2014-08-18 09:49:44,503 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] CommandMultiAsyncTasks::AttachTask: Attaching task 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 to command 9201720a-07a3-4feb-a837-fa092b7b02cc. >2014-08-18 09:49:44,523 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] Adding task 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-18 09:49:44,565 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 463eca98-08e5-47be-8be1-e620e22cc872, postZeros = false, forceDelete = false), log id: 717a18c >2014-08-18 09:49:45,356 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] FINISH, DeleteImageGroupVDSCommand, log id: 717a18c >2014-08-18 09:49:45,395 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] CommandMultiAsyncTasks::AttachTask: Attaching task 32b109af-cf6a-4d34-bd88-21cabb887e2a to command 9201720a-07a3-4feb-a837-fa092b7b02cc. >2014-08-18 09:49:45,409 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] Adding task 32b109af-cf6a-4d34-bd88-21cabb887e2a (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-18 09:49:45,438 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] BaseAsyncTask::startPollingTask: Starting to poll task 24b28d6f-3536-4ae4-ae4d-1d84586a20f1. >2014-08-18 09:49:45,438 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] BaseAsyncTask::startPollingTask: Starting to poll task 32b109af-cf6a-4d34-bd88-21cabb887e2a. >2014-08-18 09:49:45,440 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@426ccfbb), log id: 217368f >2014-08-18 09:49:45,451 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-39) [4dfbd57c] FINISH, UpdateVmDynamicDataVDSCommand, log id: 217368f >2014-08-18 09:49:50,559 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) task id 32b109af-cf6a-4d34-bd88-21cabb887e2a is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:49:50,559 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-55) task id 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:49:51,715 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-65) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: 3d7ee01d >2014-08-18 09:49:51,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-65) FINISH, FullListVdsCommand, return: [{displaySecurePort=5900, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, acpiEnable=true, custom={device_a0176251-79b2-48df-ba52-49dc7fd73498=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=a0176251-79b2-48df-ba52-49dc7fd73498, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48device_611f9766-6112-4e87-86c1-42cc5f30a38c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=611f9766-6112-4e87-86c1-42cc5f30a38c, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48device_611f9766-6112-4e87-86c1-42cc5f30a38cdevice_96c8ccf7-eae6-4354-ac32-442f636eee8c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=2d383e02-c4b2-4b55-b54c-60d73acbce48, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=11829, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@69596ff8, vmName=vm3, fileTransferEnable=true, cpuType=SandyBridge}], log id: 3d7ee01d >2014-08-18 09:49:51,736 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Received a memballoon Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-18 09:49:51,737 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Received a spice Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5900, type=graphics} >2014-08-18 09:49:52,723 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [52383d2c] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-18 09:49:52,724 ERROR [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [52383d2c] Error during CanDoActionFailure.: java.lang.NullPointerException > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.isCurrentTaskLookedFor(AsyncTaskManager.java:378) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.entityHasTasks(AsyncTaskManager.java:783) [bll.jar:] > at org.ovirt.engine.core.bll.tasks.CommandCoordinatorUtil.entityHasTasks(CommandCoordinatorUtil.java:50) [bll.jar:] > at org.ovirt.engine.core.bll.HibernateVmCommand.canDoAction(HibernateVmCommand.java:256) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view6.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.jboss.weld.servlet.ConversationPropagationFilter.doFilter(ConversationPropagationFilter.java:62) [weld-core-1.1.5.AS71.Final.jar:2012-02-10 15:31] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 09:49:52,763 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [52383d2c] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-18 09:50:00,561 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-87) task id 32b109af-cf6a-4d34-bd88-21cabb887e2a is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:00,562 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-87) task id 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:10,563 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-14) task id 32b109af-cf6a-4d34-bd88-21cabb887e2a is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:10,564 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-14) task id 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:20,486 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-40) Setting new tasks map. The map contains now 2 tasks >2014-08-18 09:50:20,564 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) task id 32b109af-cf6a-4d34-bd88-21cabb887e2a is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:20,566 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-41) task id 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:30,568 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-69) task id 32b109af-cf6a-4d34-bd88-21cabb887e2a is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:30,570 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-69) task id 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:40,572 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-96) task id 32b109af-cf6a-4d34-bd88-21cabb887e2a is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:40,573 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-96) task id 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:50,575 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) task id 32b109af-cf6a-4d34-bd88-21cabb887e2a has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:50,577 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) task id 32b109af-cf6a-4d34-bd88-21cabb887e2a has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:50,578 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) task id 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:50,579 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-22) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-18 09:50:50,580 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) task id 32b109af-cf6a-4d34-bd88-21cabb887e2a has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:50,581 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) task id 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:50,590 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) task id 32b109af-cf6a-4d34-bd88-21cabb887e2a has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:50,591 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) SPMAsyncTask::PollTask: Polling task 32b109af-cf6a-4d34-bd88-21cabb887e2a (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-18 09:50:50,616 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) BaseAsyncTask::onTaskEndSuccess: Task 32b109af-cf6a-4d34-bd88-21cabb887e2a (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-18 09:50:50,617 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-22) Task with DB Task ID 9c3151ae-23fa-44dc-bdd1-4ef983dd8ba8 and VDSM Task ID 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 is in state Polling. End action for command 9201720a-07a3-4feb-a837-fa092b7b02cc will proceed when all the entitys tasks are completed. >2014-08-18 09:50:50,619 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) task id 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-08-18 09:50:50,620 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) SPMAsyncTask::PollTask: Polling task 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-18 09:50:50,633 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) BaseAsyncTask::onTaskEndSuccess: Task 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-18 09:50:50,634 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-22) CommandAsyncTask::endActionIfNecessary: All tasks of command 9201720a-07a3-4feb-a837-fa092b7b02cc has ended -> executing endAction >2014-08-18 09:50:50,635 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-22) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 9201720a-07a3-4feb-a837-fa092b7b02cc): calling endAction . >2014-08-18 09:50:50,636 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-27) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RunVm, executionIndex: 0 >2014-08-18 09:50:50,657 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-27) Ending command successfully: org.ovirt.engine.core.bll.RunVmCommand >2014-08-18 09:50:50,680 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-27) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm completed, handling the result. >2014-08-18 09:50:50,680 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-27) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm succeeded, clearing tasks. >2014-08-18 09:50:50,691 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-27) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 32b109af-cf6a-4d34-bd88-21cabb887e2a >2014-08-18 09:50:50,691 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 32b109af-cf6a-4d34-bd88-21cabb887e2a), log id: 36f03fdd >2014-08-18 09:50:50,694 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=32b109af-cf6a-4d34-bd88-21cabb887e2a), log id: 609c2bff >2014-08-18 09:50:50,769 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) FINISH, HSMClearTaskVDSCommand, log id: 609c2bff >2014-08-18 09:50:50,770 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) FINISH, SPMClearTaskVDSCommand, log id: 36f03fdd >2014-08-18 09:50:50,782 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-27) BaseAsyncTask::removeTaskFromDB: Removed task 32b109af-cf6a-4d34-bd88-21cabb887e2a from DataBase >2014-08-18 09:50:50,782 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-27) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 >2014-08-18 09:50:50,783 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 24b28d6f-3536-4ae4-ae4d-1d84586a20f1), log id: 3ec2b647 >2014-08-18 09:50:50,785 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=24b28d6f-3536-4ae4-ae4d-1d84586a20f1), log id: 614d0f81 >2014-08-18 09:50:50,844 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) FINISH, HSMClearTaskVDSCommand, log id: 614d0f81 >2014-08-18 09:50:50,845 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-27) FINISH, SPMClearTaskVDSCommand, log id: 3ec2b647 >2014-08-18 09:50:50,856 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-27) BaseAsyncTask::removeTaskFromDB: Removed task 24b28d6f-3536-4ae4-ae4d-1d84586a20f1 from DataBase >2014-08-18 09:50:50,857 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-27) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 9201720a-07a3-4feb-a837-fa092b7b02cc >2014-08-18 09:52:20,535 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-72) Setting new tasks map. The map contains now 0 tasks >2014-08-18 09:52:20,536 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-72) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-18 16:15:21,160 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-18 16:15:21,195 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-18 16:15:21,199 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-18 16:15:21,201 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-18 16:15:21,202 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-18 16:15:21,203 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-18 16:15:21,204 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-18 16:15:21,205 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-18 16:15:21,205 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-18 16:15:21,206 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-18 16:15:21,207 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-18 16:15:21,208 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-18 16:15:21,208 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-18 16:15:21,209 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-18 16:15:21,209 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-18 16:15:21,210 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-18 16:15:21,211 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-18 16:15:21,211 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-18 16:15:21,212 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-18 16:15:21,213 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-18 16:15:21,216 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-18 16:15:21,216 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-18 16:15:21,217 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DB_USER" is "engine". >2014-08-18 16:15:21,218 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-18 16:15:21,218 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-18 16:15:21,219 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-18 16:15:21,219 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-18 16:15:21,220 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-18 16:15:21,222 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-18 16:15:21,223 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-18 16:15:21,224 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-18 16:15:21,224 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-18 16:15:21,225 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-18 16:15:21,225 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-18 16:15:21,226 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-18 16:15:21,226 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-18 16:15:21,230 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-18 16:15:21,232 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-18 16:15:21,234 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-18 16:15:21,236 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-18 16:15:21,237 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-18 16:15:21,251 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-18 16:15:21,252 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-18 16:15:21,252 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-18 16:15:21,253 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-18 16:15:21,254 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-18 16:15:21,255 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-18 16:15:21,255 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-18 16:15:21,256 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-18 16:15:21,257 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-18 16:15:21,261 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-18 16:15:21,264 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-18 16:15:21,267 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-18 16:15:21,268 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-18 16:15:21,269 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-18 16:15:21,270 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-18 16:15:21,270 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-18 16:15:21,271 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-18 16:15:21,271 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-18 16:15:21,272 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-18 16:15:21,272 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-18 16:15:21,273 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-18 16:15:21,274 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-18 16:15:21,274 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_USER" is "ovirt". >2014-08-18 16:15:21,275 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-18 16:15:21,275 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-18 16:15:21,277 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-18 16:15:21,278 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-18 16:15:21,279 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-18 16:15:21,280 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-5) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-18 16:15:22,074 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Start initializing Backend >2014-08-18 16:15:22,345 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-7) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 16:15:22,439 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-7) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 16:15:22,480 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-7) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-18 16:15:22,491 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-7) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-18 16:15:22,973 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Running ovirt-engine 3.5.0.0 >2014-08-18 16:15:22,973 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-7) Start initializing dictionaries >2014-08-18 16:15:22,976 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-7) Finished initializing dictionaries >2014-08-18 16:15:22,976 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Start initializing AuditLogCleanupManager >2014-08-18 16:15:22,977 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-18 16:15:22,999 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Finished initializing AuditLogCleanupManager >2014-08-18 16:15:22,999 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Start initializing CommandEntityCleanupManager >2014-08-18 16:15:23,000 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-18 16:15:23,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-7) Finished initializing CommandEntityCleanupManager >2014-08-18 16:15:23,003 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-7) Start initializing TagsDirector >2014-08-18 16:15:23,015 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-7) Tag root added to tree >2014-08-18 16:15:23,023 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-7) Finished initializing TagsDirector >2014-08-18 16:15:23,024 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-7) Start initializing IsoDomainListSyncronizer >2014-08-18 16:15:23,031 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-7) Finished initializing IsoDomainListSyncronizer >2014-08-18 16:15:23,098 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Completed initializing handlers >2014-08-18 16:15:23,117 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Start initializing ErrorTranslatorImpl >2014-08-18 16:15:23,124 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-18 16:15:23,125 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Finished initializing ErrorTranslatorImpl >2014-08-18 16:15:23,125 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Start initializing ErrorTranslatorImpl >2014-08-18 16:15:23,125 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-7) Finished initializing ErrorTranslatorImpl >2014-08-18 16:15:23,126 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-7) Mark incomplete jobs as UNKNOWN >2014-08-18 16:15:23,268 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-7) Start initializing JobRepositoryCleanupManager >2014-08-18 16:15:23,268 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-7) Finished initializing JobRepositoryCleanupManager >2014-08-18 16:15:23,269 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-7) Start initializing AutoRecoveryManager >2014-08-18 16:15:23,269 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-7) Finished initializing AutoRecoveryManager >2014-08-18 16:15:23,271 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-7) Start initializing ExecutionMessageDirector >2014-08-18 16:15:23,272 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-7) Finished initializing ExecutionMessageDirector >2014-08-18 16:15:23,524 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-7) Start of enabled extensions list >2014-08-18 16:15:23,524 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-7) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-18 16:15:23,525 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-7) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-18 16:15:23,526 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-7) End of enabled extensions list >2014-08-18 16:15:23,620 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-2) Start initializing DbUserCacheManager >2014-08-18 16:15:23,621 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-2) Finished initializing DbUserCacheManager >2014-08-18 16:15:23,632 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-2) Initialization of AsyncTaskManager completed successfully. >2014-08-18 16:15:23,635 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-2) Start initializing ResourceManager >2014-08-18 16:15:23,645 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-18 16:15:23,697 ERROR [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-2) Failed to initialize backend: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [select * from getallfromvms(?, ?)]; nested exception is org.postgresql.util.PSQLException: The column name memory_usage_history was not found in this ResultSet. > at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:98) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:603) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VmDAODbFacadeImpl.getAll(VmDAODbFacadeImpl.java:201) [dal.jar:] > at org.ovirt.engine.core.dao.VmDAODbFacadeImpl.getAll(VmDAODbFacadeImpl.java:196) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.init(ResourceManager.java:82) [vdsbroker.jar:] > at org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean.create(InitBackendServicesOnStartupBean.java:66) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] >Caused by: org.postgresql.util.PSQLException: The column name memory_usage_history was not found in this ResultSet. > at org.postgresql.jdbc2.AbstractJdbc2ResultSet.findColumn(AbstractJdbc2ResultSet.java:2542) > at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getObject(AbstractJdbc2ResultSet.java:2530) > at org.jboss.jca.adapters.jdbc.WrappedResultSet.getObject(WrappedResultSet.java:1199) > at org.ovirt.engine.core.dao.VmStatisticsDaoDbFacadeImpl$VmStatisticsRowMapper.mapRow(VmStatisticsDaoDbFacadeImpl.java:77) [dal.jar:] > at org.ovirt.engine.core.dao.VmStatisticsDaoDbFacadeImpl$VmStatisticsRowMapper.mapRow(VmStatisticsDaoDbFacadeImpl.java:63) [dal.jar:] > at org.ovirt.engine.core.dao.VmDAODbFacadeImpl$VMRowMapper.mapRow(VmDAODbFacadeImpl.java:299) [dal.jar:] > at org.ovirt.engine.core.dao.VmDAODbFacadeImpl$VMRowMapper.mapRow(VmDAODbFacadeImpl.java:290) [dal.jar:] > at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:92) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:1) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:649) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.1.RELEASE] > ... 47 more > >2014-08-18 16:17:25,827 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-18 16:17:25,836 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-18 16:17:25,847 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-18 16:17:25,848 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-18 16:17:25,850 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-18 16:17:25,851 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-18 16:17:25,853 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-18 16:17:25,857 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-18 16:17:25,858 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-18 16:17:25,861 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-18 16:17:25,864 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-18 16:17:25,865 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-18 16:17:25,865 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-18 16:17:25,866 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-18 16:17:25,866 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-18 16:17:25,867 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-18 16:17:25,870 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-18 16:17:25,871 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-18 16:17:25,872 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-18 16:17:25,872 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-18 16:17:25,873 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-18 16:17:25,874 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-18 16:17:25,875 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DB_USER" is "engine". >2014-08-18 16:17:25,875 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-18 16:17:25,876 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-18 16:17:25,877 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-18 16:17:25,877 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-18 16:17:25,878 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-18 16:17:25,878 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-18 16:17:25,879 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-18 16:17:25,879 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-18 16:17:25,879 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-18 16:17:25,880 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-18 16:17:25,880 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-18 16:17:25,881 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-18 16:17:25,882 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-18 16:17:25,882 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-18 16:17:25,883 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-18 16:17:25,884 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-18 16:17:25,885 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-18 16:17:25,886 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-18 16:17:25,886 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-18 16:17:25,887 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-18 16:17:25,887 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-18 16:17:25,888 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-18 16:17:25,888 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-18 16:17:25,889 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-18 16:17:25,889 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-18 16:17:25,890 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-18 16:17:25,890 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-18 16:17:25,891 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-18 16:17:25,891 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-18 16:17:25,892 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-18 16:17:25,892 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-18 16:17:25,893 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-18 16:17:25,896 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-18 16:17:25,898 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-18 16:17:25,901 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-18 16:17:25,902 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-18 16:17:25,902 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-18 16:17:25,902 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-18 16:17:25,903 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-18 16:17:25,904 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-18 16:17:25,904 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_USER" is "ovirt". >2014-08-18 16:17:25,905 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-18 16:17:25,905 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-18 16:17:25,906 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-18 16:17:25,906 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-18 16:17:25,906 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-18 16:17:25,907 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-3) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-18 16:17:26,163 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) Start initializing Backend >2014-08-18 16:17:26,286 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-5) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 16:17:26,362 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-5) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 16:17:26,388 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-5) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-18 16:17:26,398 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-5) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-18 16:17:26,522 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) Running ovirt-engine 3.5.0.0 >2014-08-18 16:17:26,522 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-5) Start initializing dictionaries >2014-08-18 16:17:26,526 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-5) Finished initializing dictionaries >2014-08-18 16:17:26,526 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Start initializing AuditLogCleanupManager >2014-08-18 16:17:26,527 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-18 16:17:26,542 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Finished initializing AuditLogCleanupManager >2014-08-18 16:17:26,542 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Start initializing CommandEntityCleanupManager >2014-08-18 16:17:26,543 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-18 16:17:26,544 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-5) Finished initializing CommandEntityCleanupManager >2014-08-18 16:17:26,546 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-5) Start initializing TagsDirector >2014-08-18 16:17:26,555 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-5) Tag root added to tree >2014-08-18 16:17:26,562 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-5) Finished initializing TagsDirector >2014-08-18 16:17:26,563 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-5) Start initializing IsoDomainListSyncronizer >2014-08-18 16:17:26,569 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-5) Finished initializing IsoDomainListSyncronizer >2014-08-18 16:17:26,625 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) Completed initializing handlers >2014-08-18 16:17:26,643 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-5) Start initializing ErrorTranslatorImpl >2014-08-18 16:17:26,656 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-5) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-18 16:17:26,658 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-5) Finished initializing ErrorTranslatorImpl >2014-08-18 16:17:26,658 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-5) Start initializing ErrorTranslatorImpl >2014-08-18 16:17:26,659 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-5) Finished initializing ErrorTranslatorImpl >2014-08-18 16:17:26,659 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-5) Mark incomplete jobs as UNKNOWN >2014-08-18 16:17:26,711 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-5) Start initializing JobRepositoryCleanupManager >2014-08-18 16:17:26,711 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-5) Finished initializing JobRepositoryCleanupManager >2014-08-18 16:17:26,712 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-5) Start initializing AutoRecoveryManager >2014-08-18 16:17:26,713 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-5) Finished initializing AutoRecoveryManager >2014-08-18 16:17:26,715 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-5) Start initializing ExecutionMessageDirector >2014-08-18 16:17:26,717 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-5) Finished initializing ExecutionMessageDirector >2014-08-18 16:17:26,761 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-5) Start of enabled extensions list >2014-08-18 16:17:26,762 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-5) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-18 16:17:26,763 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-5) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-18 16:17:26,764 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-5) End of enabled extensions list >2014-08-18 16:17:26,813 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-5) Start initializing DbUserCacheManager >2014-08-18 16:17:26,814 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-5) Finished initializing DbUserCacheManager >2014-08-18 16:17:26,823 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-5) Initialization of AsyncTaskManager completed successfully. >2014-08-18 16:17:26,825 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-5) Start initializing ResourceManager >2014-08-18 16:17:26,841 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-18 16:17:26,898 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-5) Entered VdsManager constructor >2014-08-18 16:17:26,936 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-5) Initialize vdsBroker (galactica,54,321) >2014-08-18 16:17:26,974 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-5) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-18 16:17:26,974 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-5) Entered VdsManager constructor >2014-08-18 16:17:26,985 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-5) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-18 16:17:27,049 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-5) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-18 16:17:27,067 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-5) Finished initializing ResourceManager >2014-08-18 16:17:27,070 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-5) Initialization of OvfDataUpdater completed successfully. >2014-08-18 16:17:27,071 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) Start scheduling to enable vds load balancer >2014-08-18 16:17:27,072 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) Finished scheduling to enable vds load balancer >2014-08-18 16:17:27,072 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) Start HA Reservation check >2014-08-18 16:17:27,073 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) Finished HA Reservation check >2014-08-18 16:17:27,080 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-5) Init VM custom properties utilities >2014-08-18 16:17:27,081 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-5) Init device custom properties utilities >2014-08-18 16:17:27,086 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) Initializing Scheduling manager >2014-08-18 16:17:27,103 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-18 16:17:27,104 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-18 16:17:27,105 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-18 16:17:27,110 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) External scheduler disabled, discovery skipped >2014-08-18 16:17:27,110 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-5) Initialized Scheduling manager >2014-08-18 16:17:27,111 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-5) Initializing DWH Heart Beat >2014-08-18 16:17:27,112 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-5) DWH Heart Beat initialized >2014-08-18 16:17:30,009 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h2 >2014-08-18 16:17:31,760 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 327782d8 >2014-08-18 16:17:31,917 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GetHardwareInfoVDSCommand, log id: 327782d8 >2014-08-18 16:17:31,921 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-18 16:17:31,926 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: UNASSIGNED not exist in string table >2014-08-18 16:17:31,927 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-18 16:17:31,927 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-18 16:17:31,929 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-18 16:17:31,930 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-18 16:17:31,931 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-18 16:17:31,931 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-18 16:17:31,932 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-18 16:17:31,933 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-18 16:17:31,934 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-18 16:17:31,934 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-18 16:17:31,935 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: DWH_STOPPED not exist in string table >2014-08-18 16:17:31,935 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: DWH_STARTED not exist in string table >2014-08-18 16:17:31,935 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: DWH_ERROR not exist in string table >2014-08-18 16:17:31,936 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-08-18 16:17:31,936 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-08-18 16:17:31,937 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-08-18 16:17:31,937 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-08-18 16:17:31,937 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-08-18 16:17:31,938 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-08-18 16:17:31,938 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-18 16:17:31,938 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-18 16:17:31,939 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-18 16:17:31,939 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-18 16:17:32,210 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-18 16:17:32,211 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is running with disabled SELinux. >2014-08-18 16:17:32,903 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: h1 >2014-08-18 16:17:33,074 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-7) [475e00f] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-18 16:17:37,115 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-23) [323cc39f] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-18 16:17:37,125 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [323cc39f] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-18 16:17:37,131 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [323cc39f] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-18 16:17:37,427 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [323cc39f] Correlation ID: 323cc39f, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-18 16:17:37,667 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) [323cc39f] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-18 16:17:37,711 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) [323cc39f] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-18 16:17:37,742 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [323cc39f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-18 16:17:37,759 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [323cc39f] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 4d420393 >2014-08-18 16:17:38,346 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [323cc39f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 28 GB of free space >2014-08-18 16:17:38,379 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [323cc39f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 28 GB of free space >2014-08-18 16:17:38,380 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [323cc39f] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-18 16:17:38,388 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [323cc39f] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: ec2e609 >2014-08-18 16:17:38,552 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [323cc39f] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: ec2e609 >2014-08-18 16:17:38,553 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [323cc39f] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 4d420393 >2014-08-18 16:17:38,554 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [323cc39f] Discovered no tasks on Storage Pool Default >2014-08-18 16:18:05,909 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-4) Running command: LoginAdminUserCommand internal: false. >2014-08-18 16:18:05,958 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-18 16:18:09,175 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-5) Running command: LoginUserCommand internal: false. >2014-08-18 16:20:50,900 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (http--0.0.0.0-8080-4) [6590a448] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-18 16:20:51,176 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-6) [6590a448] Running command: HibernateVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-18 16:20:51,206 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-6) [6590a448] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@80dab90b), log id: 6a134399 >2014-08-18 16:20:51,208 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-6) [6590a448] FINISH, UpdateVmDynamicDataVDSCommand, log id: 6a134399 >2014-08-18 16:20:51,222 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-6) [6590a448] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 4aa5a3b5-6d38-4aad-b5cd-040082eb7a4f, imageSizeInBytes = 1350565888, volumeFormat = RAW, newImageId = 9b70236e-5068-4717-8561-0d158c2d7a94, newImageDescription = ), log id: 6044b5c1 >2014-08-18 16:20:51,463 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-6) [6590a448] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-18 16:20:52,122 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-6) [6590a448] FINISH, CreateImageVDSCommand, return: 9b70236e-5068-4717-8561-0d158c2d7a94, log id: 6044b5c1 >2014-08-18 16:20:52,180 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-6) [6590a448] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 5b5026c5-2712-469c-8636-6fe94cd0ab42 >2014-08-18 16:20:52,181 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-6) [6590a448] CommandMultiAsyncTasks::AttachTask: Attaching task 0cc3e7f6-a292-4795-bdc3-ff83dd328176 to command 5b5026c5-2712-469c-8636-6fe94cd0ab42. >2014-08-18 16:20:52,208 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-6) [6590a448] Adding task 0cc3e7f6-a292-4795-bdc3-ff83dd328176 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-18 16:20:52,229 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-6) [6590a448] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 1b44587c-fcd1-4b41-a07f-55d6cf4626d9, imageSizeInBytes = 10240, volumeFormat = COW, newImageId = ae165814-de5a-45bf-8636-2cea3e3805d9, newImageDescription = ), log id: 118b9a7 >2014-08-18 16:20:52,233 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-6) [6590a448] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-18 16:20:53,318 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (org.ovirt.thread.pool-8-thread-6) [6590a448] FINISH, CreateImageVDSCommand, return: ae165814-de5a-45bf-8636-2cea3e3805d9, log id: 118b9a7 >2014-08-18 16:20:53,338 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-6) [6590a448] CommandMultiAsyncTasks::AttachTask: Attaching task 7238971d-da29-4e58-93ee-5d148535f22c to command 5b5026c5-2712-469c-8636-6fe94cd0ab42. >2014-08-18 16:20:53,362 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-6) [6590a448] Adding task 7238971d-da29-4e58-93ee-5d148535f22c (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-18 16:20:53,372 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-6) [6590a448] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@7c4e1944), log id: 7a3222a9 >2014-08-18 16:20:53,378 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-6) [6590a448] FINISH, UpdateVmDynamicDataVDSCommand, log id: 7a3222a9 >2014-08-18 16:20:53,478 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-6) [6590a448] Correlation ID: 6590a448, Job ID: c47719ab-a2b8-4684-b44c-8121fd4703c5, Call Stack: null, Custom Event ID: -1, Message: Suspending VM vm3 was initiated by User admin (Host: h1). >2014-08-18 16:20:53,479 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-6) [6590a448] BaseAsyncTask::startPollingTask: Starting to poll task 0cc3e7f6-a292-4795-bdc3-ff83dd328176. >2014-08-18 16:20:53,479 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-6) [6590a448] BaseAsyncTask::startPollingTask: Starting to poll task 7238971d-da29-4e58-93ee-5d148535f22c. >2014-08-18 16:20:53,480 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-6) [6590a448] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-18 16:20:56,873 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-65) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-08-18 16:20:57,032 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-65) SPMAsyncTask::PollTask: Polling task 0cc3e7f6-a292-4795-bdc3-ff83dd328176 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-18 16:20:57,088 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-65) BaseAsyncTask::onTaskEndSuccess: Task 0cc3e7f6-a292-4795-bdc3-ff83dd328176 (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-18 16:20:57,089 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-65) Task with DB Task ID 0a4d4746-d129-413d-a978-38dec1577e76 and VDSM Task ID 7238971d-da29-4e58-93ee-5d148535f22c is in state Polling. End action for command 5b5026c5-2712-469c-8636-6fe94cd0ab42 will proceed when all the entitys tasks are completed. >2014-08-18 16:20:57,090 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-65) SPMAsyncTask::PollTask: Polling task 7238971d-da29-4e58-93ee-5d148535f22c (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-18 16:20:57,097 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-65) BaseAsyncTask::onTaskEndSuccess: Task 7238971d-da29-4e58-93ee-5d148535f22c (Parent Command HibernateVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-18 16:20:57,097 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-65) CommandAsyncTask::endActionIfNecessary: All tasks of command 5b5026c5-2712-469c-8636-6fe94cd0ab42 has ended -> executing endAction >2014-08-18 16:20:57,098 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-65) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 5b5026c5-2712-469c-8636-6fe94cd0ab42): calling endAction . >2014-08-18 16:20:57,099 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-13) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction HibernateVm, executionIndex: 0 >2014-08-18 16:20:57,155 INFO [org.ovirt.engine.core.bll.HibernateVmCommand] (org.ovirt.thread.pool-8-thread-13) Ending command successfully: org.ovirt.engine.core.bll.HibernateVmCommand >2014-08-18 16:20:57,158 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-13) START, HibernateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,4aa5a3b5-6d38-4aad-b5cd-040082eb7a4f,9b70236e-5068-4717-8561-0d158c2d7a94,1b44587c-fcd1-4b41-a07f-55d6cf4626d9,ae165814-de5a-45bf-8636-2cea3e3805d9), log id: 243475f8 >2014-08-18 16:20:57,236 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-13) START, HibernateBrokerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, hibernationVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,4aa5a3b5-6d38-4aad-b5cd-040082eb7a4f,9b70236e-5068-4717-8561-0d158c2d7a94,1b44587c-fcd1-4b41-a07f-55d6cf4626d9,ae165814-de5a-45bf-8636-2cea3e3805d9), log id: 68cfa395 >2014-08-18 16:20:57,396 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HibernateBrokerVDSCommand] (org.ovirt.thread.pool-8-thread-13) FINISH, HibernateBrokerVDSCommand, log id: 68cfa395 >2014-08-18 16:20:57,412 INFO [org.ovirt.engine.core.vdsbroker.HibernateVDSCommand] (org.ovirt.thread.pool-8-thread-13) FINISH, HibernateVDSCommand, log id: 243475f8 >2014-08-18 16:20:57,414 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-13) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm completed, handling the result. >2014-08-18 16:20:57,415 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-13) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type HibernateVm succeeded, clearing tasks. >2014-08-18 16:20:57,428 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-13) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 0cc3e7f6-a292-4795-bdc3-ff83dd328176 >2014-08-18 16:20:57,429 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 0cc3e7f6-a292-4795-bdc3-ff83dd328176), log id: 31f5758c >2014-08-18 16:20:57,432 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=0cc3e7f6-a292-4795-bdc3-ff83dd328176), log id: 69dfb97c >2014-08-18 16:20:57,754 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) FINISH, HSMClearTaskVDSCommand, log id: 69dfb97c >2014-08-18 16:20:57,755 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) FINISH, SPMClearTaskVDSCommand, log id: 31f5758c >2014-08-18 16:20:57,776 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-13) BaseAsyncTask::removeTaskFromDB: Removed task 0cc3e7f6-a292-4795-bdc3-ff83dd328176 from DataBase >2014-08-18 16:20:57,776 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-13) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 7238971d-da29-4e58-93ee-5d148535f22c >2014-08-18 16:20:57,777 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 7238971d-da29-4e58-93ee-5d148535f22c), log id: 603e2737 >2014-08-18 16:20:57,778 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=7238971d-da29-4e58-93ee-5d148535f22c), log id: 1080bd3e >2014-08-18 16:20:58,062 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) FINISH, HSMClearTaskVDSCommand, log id: 1080bd3e >2014-08-18 16:20:58,062 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-13) FINISH, SPMClearTaskVDSCommand, log id: 603e2737 >2014-08-18 16:20:58,082 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-13) BaseAsyncTask::removeTaskFromDB: Removed task 7238971d-da29-4e58-93ee-5d148535f22c from DataBase >2014-08-18 16:20:58,083 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-13) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 5b5026c5-2712-469c-8636-6fe94cd0ab42 >2014-08-18 16:21:00,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-76) START, DestroyVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, force=false, secondsToWait=0, gracefully=false, reason=), log id: 2be206b1 >2014-08-18 16:21:00,743 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-76) FINISH, DestroyVDSCommand, log id: 2be206b1 >2014-08-18 16:21:00,807 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-76) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm3 on Host h1 is suspended. >2014-08-18 16:21:00,808 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) VM vm3 (5b1ee280-4251-489a-899c-8e9dac73c327) is running in db and not running in VDS h1 >2014-08-18 16:21:00,809 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-76) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: 75860a94 >2014-08-18 16:21:00,964 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-76) FINISH, FullListVdsCommand, return: [], log id: 75860a94 >2014-08-18 16:21:01,024 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-24) [1c219df7] Running command: ProcessDownVmCommand internal: true. >2014-08-18 16:21:26,512 INFO [org.ovirt.engine.core.bll.RunVmCommand] (http--0.0.0.0-8080-4) [11956b7a] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-18 16:21:26,595 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [11956b7a] START, IsVmDuringInitiatingVDSCommand( vmId = 5b1ee280-4251-489a-899c-8e9dac73c327), log id: 5e5f953d >2014-08-18 16:21:26,596 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (http--0.0.0.0-8080-4) [11956b7a] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 5e5f953d >2014-08-18 16:21:26,706 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-1) [11956b7a] Running command: RunVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-08-18 16:21:26,802 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-1) [11956b7a] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = ISCSI, connectionList = [{ id: bd327084-78e4-4b84-9a74-9cea2d43c971, connection: 10.35.16.25, iqn: iqn.2011-05.com.redhat:achub-iscsi, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 63e3d1d7 >2014-08-18 16:21:27,732 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-1) [11956b7a] FINISH, ConnectStorageServerVDSCommand, return: {bd327084-78e4-4b84-9a74-9cea2d43c971=0}, log id: 63e3d1d7 >2014-08-18 16:21:27,744 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-1) [11956b7a] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@e722ad5), log id: 36415bc8 >2014-08-18 16:21:27,757 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-1) [11956b7a] FINISH, UpdateVmDynamicDataVDSCommand, log id: 36415bc8 >2014-08-18 16:21:27,778 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-1) [11956b7a] START, CreateVmVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, vm=VM [vm3]), log id: 6e74c7c >2014-08-18 16:21:27,789 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-1) [11956b7a] START, CreateVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, vm=VM [vm3]), log id: 2535b788 >2014-08-18 16:21:27,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-1) [11956b7a] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,hiberVolHandle=7158eae9-4886-4af7-af0b-d668ecee9e4f,00000002-0002-0002-0002-00000000037f,4aa5a3b5-6d38-4aad-b5cd-040082eb7a4f,9b70236e-5068-4717-8561-0d158c2d7a94,1b44587c-fcd1-4b41-a07f-55d6cf4626d9,ae165814-de5a-45bf-8636-2cea3e3805d9,transparentHugePages=true,vmId=5b1ee280-4251-489a-899c-8e9dac73c327,acpiEnable=true,custom={device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_63575f20-b9d6-4a3f-ae01-4ae43e7e8d1c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=63575f20-b9d6-4a3f-ae01-4ae43e7e8d1c, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_63575f20-b9d6-4a3f-ae01-4ae43e7e8d1cdevice_55fbcdf6-19ce-4417-9c03-fb1b500e239bdevice_8393b62a-ce6c-4688-8e1b-9066d1f1d217device_bc2363f6-63f6-474e-8f5c-645547171f71=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=bc2363f6-63f6-474e-8f5c-645547171f71, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_96c8ccf7-eae6-4354-ac32-442f636eee8c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_63575f20-b9d6-4a3f-ae01-4ae43e7e8d1cdevice_55fbcdf6-19ce-4417-9c03-fb1b500e239bdevice_8393b62a-ce6c-4688-8e1b-9066d1f1d217=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=8393b62a-ce6c-4688-8e1b-9066d1f1d217, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_96c8ccf7-eae6-4354-ac32-442f636eee8cdevice_63575f20-b9d6-4a3f-ae01-4ae43e7e8d1cdevice_55fbcdf6-19ce-4417-9c03-fb1b500e239b=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=55fbcdf6-19ce-4417-9c03-fb1b500e239b, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=pc-1.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=interleave},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=f09597d8-8cc8-4549-863f-b96975014b01}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=f76be01f-4e52-4d67-8e1b-0856337e9929}, {shared=false, index=0, volumeID=26308abd-8540-410e-8018-58cd2157084d, propagateErrors=off, format=cow, type=disk, iface=virtio, bootOrder=1, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=7158eae9-4886-4af7-af0b-d668ecee9e4f, imageID=b5576e45-7f13-465f-b26f-05d52faf28d1, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000037f, readonly=false, deviceId=b5576e45-7f13-465f-b26f-05d52faf28d1}, {shared=false, iface=virtio, GUID=3600144f09dbd050000004e1994c60005, address={bus=0x00, domain=0x0000, slot=0x07, type=pci, function=0x0}, specParams={}, optional=false, propagateErrors=off, device=lun, format=raw, type=disk, readonly=false, deviceId=bdca3776-6597-4323-84f1-4ea75f7d269d}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:16:01:53, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=ovirtmgmt, deviceId=68e93502-4ad4-42a0-9a23-dfa3f67090f5}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=403887de-d0b8-4108-8b43-b4b5d5413859}, {address={bus=0x00, domain=0x0000, slot=0x05, type=pci, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c}],vmName=vm3,cpuType=SandyBridge,fileTransferEnable=true >2014-08-18 16:21:28,431 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-1) [11956b7a] FINISH, CreateVDSCommand, log id: 2535b788 >2014-08-18 16:21:28,486 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-1) [11956b7a] FINISH, CreateVmVDSCommand, return: RestoringState, log id: 6e74c7c >2014-08-18 16:21:28,487 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-1) [11956b7a] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-18 16:21:28,503 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-1) [11956b7a] Correlation ID: 11956b7a, Job ID: 74dc01f2-1382-4b20-b49d-cb721d95b80b, Call Stack: null, Custom Event ID: -1, Message: VM vm3 was started by admin (Host: h1). >2014-08-18 16:21:30,918 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) VM vm3 5b1ee280-4251-489a-899c-8e9dac73c327 moved from RestoringState --> Up >2014-08-18 16:21:30,920 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-60) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: 70170aa3 >2014-08-18 16:21:31,229 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-60) FINISH, FullListVdsCommand, return: [{displaySecurePort=5900, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, acpiEnable=true, custom={device_a0176251-79b2-48df-ba52-49dc7fd73498=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=a0176251-79b2-48df-ba52-49dc7fd73498, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48device_611f9766-6112-4e87-86c1-42cc5f30a38c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=611f9766-6112-4e87-86c1-42cc5f30a38c, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48device_611f9766-6112-4e87-86c1-42cc5f30a38cdevice_96c8ccf7-eae6-4354-ac32-442f636eee8c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=2d383e02-c4b2-4b55-b54c-60d73acbce48, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=5154, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@3fdb471, vmName=vm3, fileTransferEnable=true, cpuType=SandyBridge}], log id: 70170aa3 >2014-08-18 16:21:31,247 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Received a memballoon Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-18 16:21:31,249 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Received a spice Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5900, type=graphics} >2014-08-18 16:21:31,453 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-60) Correlation ID: 11956b7a, Job ID: 74dc01f2-1382-4b20-b49d-cb721d95b80b, Call Stack: null, Custom Event ID: -1, Message: VM vm3 started on Host h1 >2014-08-18 16:21:31,463 INFO [org.ovirt.engine.core.bll.RemoveVmHibernationVolumesCommand] (DefaultQuartzScheduler_Worker-60) [20a0389e] Running command: RemoveVmHibernationVolumesCommand internal: true. >2014-08-18 16:21:31,545 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-60) [20a0389e] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 4aa5a3b5-6d38-4aad-b5cd-040082eb7a4f, postZeros = false, forceDelete = false), log id: 4d2f09c5 >2014-08-18 16:21:32,575 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-60) [20a0389e] FINISH, DeleteImageGroupVDSCommand, log id: 4d2f09c5 >2014-08-18 16:21:32,623 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-60) [20a0389e] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command ace51e2f-6220-406b-aec4-ba2f7f0b7a2c >2014-08-18 16:21:32,624 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-60) [20a0389e] CommandMultiAsyncTasks::AttachTask: Attaching task fc6948f4-838e-4171-9211-c49191c57aa5 to command ace51e2f-6220-406b-aec4-ba2f7f0b7a2c. >2014-08-18 16:21:32,645 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-60) [20a0389e] Adding task fc6948f4-838e-4171-9211-c49191c57aa5 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-18 16:21:32,687 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-60) [20a0389e] START, DeleteImageGroupVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 1b44587c-fcd1-4b41-a07f-55d6cf4626d9, postZeros = false, forceDelete = false), log id: 49c36b0a >2014-08-18 16:21:33,482 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (DefaultQuartzScheduler_Worker-60) [20a0389e] FINISH, DeleteImageGroupVDSCommand, log id: 49c36b0a >2014-08-18 16:21:33,517 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-60) [20a0389e] CommandMultiAsyncTasks::AttachTask: Attaching task f8d9f973-9357-4d5a-8233-ee774117c150 to command ace51e2f-6220-406b-aec4-ba2f7f0b7a2c. >2014-08-18 16:21:33,531 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-60) [20a0389e] Adding task f8d9f973-9357-4d5a-8233-ee774117c150 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-18 16:21:33,559 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-60) [20a0389e] BaseAsyncTask::startPollingTask: Starting to poll task fc6948f4-838e-4171-9211-c49191c57aa5. >2014-08-18 16:21:33,559 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-60) [20a0389e] BaseAsyncTask::startPollingTask: Starting to poll task f8d9f973-9357-4d5a-8233-ee774117c150. >2014-08-18 16:21:33,561 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-60) [20a0389e] START, UpdateVmDynamicDataVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@1f2a9e1f), log id: 47148f79 >2014-08-18 16:21:33,573 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (DefaultQuartzScheduler_Worker-60) [20a0389e] FINISH, UpdateVmDynamicDataVDSCommand, log id: 47148f79 >2014-08-18 16:21:37,100 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-76) Polling and updating Async Tasks: 4 tasks, 2 tasks to poll now >2014-08-18 16:21:37,255 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-76) SPMAsyncTask::PollTask: Polling task f8d9f973-9357-4d5a-8233-ee774117c150 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-18 16:21:37,291 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-76) BaseAsyncTask::onTaskEndSuccess: Task f8d9f973-9357-4d5a-8233-ee774117c150 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-18 16:21:37,292 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-76) Task with DB Task ID 35f9cb04-884b-49f2-89c4-38bf37a844c2 and VDSM Task ID fc6948f4-838e-4171-9211-c49191c57aa5 is in state Polling. End action for command ace51e2f-6220-406b-aec4-ba2f7f0b7a2c will proceed when all the entitys tasks are completed. >2014-08-18 16:21:37,292 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-76) SPMAsyncTask::PollTask: Polling task fc6948f4-838e-4171-9211-c49191c57aa5 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-18 16:21:37,300 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-76) BaseAsyncTask::onTaskEndSuccess: Task fc6948f4-838e-4171-9211-c49191c57aa5 (Parent Command RunVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-18 16:21:37,300 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-76) CommandAsyncTask::endActionIfNecessary: All tasks of command ace51e2f-6220-406b-aec4-ba2f7f0b7a2c has ended -> executing endAction >2014-08-18 16:21:37,300 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-76) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: ace51e2f-6220-406b-aec4-ba2f7f0b7a2c): calling endAction . >2014-08-18 16:21:37,301 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-19) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction RunVm, executionIndex: 0 >2014-08-18 16:21:37,342 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-19) Ending command successfully: org.ovirt.engine.core.bll.RunVmCommand >2014-08-18 16:21:37,344 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-19) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm completed, handling the result. >2014-08-18 16:21:37,344 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-19) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type RunVm succeeded, clearing tasks. >2014-08-18 16:21:37,358 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-19) SPMAsyncTask::ClearAsyncTask: Attempting to clear task f8d9f973-9357-4d5a-8233-ee774117c150 >2014-08-18 16:21:37,358 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-19) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = f8d9f973-9357-4d5a-8233-ee774117c150), log id: 54eff728 >2014-08-18 16:21:37,361 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-19) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=f8d9f973-9357-4d5a-8233-ee774117c150), log id: 7a82e3da >2014-08-18 16:21:37,578 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-19) FINISH, HSMClearTaskVDSCommand, log id: 7a82e3da >2014-08-18 16:21:37,579 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-19) FINISH, SPMClearTaskVDSCommand, log id: 54eff728 >2014-08-18 16:21:37,590 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-19) BaseAsyncTask::removeTaskFromDB: Removed task f8d9f973-9357-4d5a-8233-ee774117c150 from DataBase >2014-08-18 16:21:37,590 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-19) SPMAsyncTask::ClearAsyncTask: Attempting to clear task fc6948f4-838e-4171-9211-c49191c57aa5 >2014-08-18 16:21:37,591 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-19) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = fc6948f4-838e-4171-9211-c49191c57aa5), log id: 77e921a >2014-08-18 16:21:37,592 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-19) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=fc6948f4-838e-4171-9211-c49191c57aa5), log id: 6a9ecce >2014-08-18 16:21:37,985 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-19) FINISH, HSMClearTaskVDSCommand, log id: 6a9ecce >2014-08-18 16:21:37,985 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-19) FINISH, SPMClearTaskVDSCommand, log id: 77e921a >2014-08-18 16:21:38,136 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-19) BaseAsyncTask::removeTaskFromDB: Removed task fc6948f4-838e-4171-9211-c49191c57aa5 from DataBase >2014-08-18 16:21:38,137 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-19) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity ace51e2f-6220-406b-aec4-ba2f7f0b7a2c >2014-08-18 16:21:43,575 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-96) START, FullListVdsCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77], vmIds=[5b1ee280-4251-489a-899c-8e9dac73c327]), log id: 303b7697 >2014-08-18 16:21:43,888 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-96) FINISH, FullListVdsCommand, return: [{displaySecurePort=5900, kvmEnable=true, nicModel=rtl8139,pv, afterMigrationStatus=, keyboardLayout=en-us, displayIp=10.35.0.105, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=5b1ee280-4251-489a-899c-8e9dac73c327, acpiEnable=true, custom={device_a0176251-79b2-48df-ba52-49dc7fd73498=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=a0176251-79b2-48df-ba52-49dc7fd73498, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48device_611f9766-6112-4e87-86c1-42cc5f30a38c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=611f9766-6112-4e87-86c1-42cc5f30a38c, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48device_611f9766-6112-4e87-86c1-42cc5f30a38cdevice_96c8ccf7-eae6-4354-ac32-442f636eee8c=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=96c8ccf7-eae6-4354-ac32-442f636eee8c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_a0176251-79b2-48df-ba52-49dc7fd73498device_2d383e02-c4b2-4b55-b54c-60d73acbce48=VmDevice {vmId=5b1ee280-4251-489a-899c-8e9dac73c327, deviceId=2d383e02-c4b2-4b55-b54c-60d73acbce48, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=1024, smp=1, displayPort=-1, emulatedMachine=pc-1.0, vmType=kvm, status=Up, memGuaranteedSize=1024, display=qxl, pid=5154, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@fe6cc3e, vmName=vm3, fileTransferEnable=true, cpuType=SandyBridge}], log id: 303b7697 >2014-08-18 16:21:43,903 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Received a memballoon Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {alias=balloon0, specParams={model=none}, target=1048576, device=memballoon, type=balloon} >2014-08-18 16:21:43,906 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Received a spice Device without an address when processing VM 5b1ee280-4251-489a-899c-8e9dac73c327 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.35.0.105}, device=spice, tlsPort=5900, type=graphics} >2014-08-18 16:22:26,819 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-7) [475e00f] Setting new tasks map. The map contains now 2 tasks >2014-08-18 16:22:56,820 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-96) Setting new tasks map. The map contains now 0 tasks >2014-08-18 16:22:56,821 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-96) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-18 16:40:26,752 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [1ee8d094] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-18 16:40:26,770 ERROR [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [1ee8d094] Error during CanDoActionFailure.: java.lang.NullPointerException > at java.util.AbstractCollection.addAll(AbstractCollection.java:333) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.bll.ImagesHandler.getAllStorageIdsForImageIds(ImagesHandler.java:505) [bll.jar:] > at org.ovirt.engine.core.bll.VmCommand.createMultipleStorageDomainsValidator(VmCommand.java:556) [bll.jar:] > at org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.validateStorageDomains(CreateAllSnapshotsFromVmCommand.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.canDoAction(CreateAllSnapshotsFromVmCommand.java:524) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 16:40:26,797 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [1ee8d094] Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-18 16:40:50,435 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-5) [7b4943ec] Lock Acquired to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 16:40:50,450 ERROR [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-5) [7b4943ec] Error during CanDoActionFailure.: java.lang.NullPointerException > at java.util.AbstractCollection.addAll(AbstractCollection.java:333) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.bll.ImagesHandler.getAllStorageIdsForImageIds(ImagesHandler.java:505) [bll.jar:] > at org.ovirt.engine.core.bll.VmCommand.createMultipleStorageDomainsValidator(VmCommand.java:556) [bll.jar:] > at org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.validateStorageDomains(CreateAllSnapshotsFromVmCommand.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.canDoAction(CreateAllSnapshotsFromVmCommand.java:524) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 16:40:50,483 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-5) [7b4943ec] Lock freed to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 16:41:35,087 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-100) Command ListVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException >2014-08-18 16:41:35,089 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h1 is not responding. It will stay in Connecting state for a grace period of $81 seconds and after that an attempt to fence the host will be issued. >2014-08-18 16:41:35,094 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:410) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.fetchRunningVms(VdsUpdateRunTimeInfo.java:971) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:920) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:638) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:485) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.util.concurrent.TimeoutException > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:119) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy.invoke(XmlRpcUtils.java:200) [vdsbroker.jar:] > at com.sun.proxy.$Proxy83.list(Unknown Source) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper.list(VdsServerWrapper.java:127) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand.executeVdsBrokerCommand(ListVDSCommand.java:23) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > ... 14 more > >2014-08-18 16:41:35,175 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.util.concurrent.TimeoutException >2014-08-18 16:41:35,987 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-2) Command SpmStatusVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException >2014-08-18 16:41:35,989 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-2) [7b5b7e93] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-18 16:41:35,992 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-2) [7b5b7e93] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-18 16:41:35,994 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-2) [7b5b7e93] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-18 16:41:36,027 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-2) [7b5b7e93] Correlation ID: 7b5b7e93, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host h1, Error: Network error during communication with the Host.). >2014-08-18 16:41:38,485 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-33) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 78ace919 >2014-08-18 16:41:38,492 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-33) FINISH, GetHardwareInfoVDSCommand, log id: 78ace919 >2014-08-18 16:41:38,494 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-33) Host h1 is running with disabled SELinux. >2014-08-18 16:41:38,545 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-33) [19371e17] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-18 16:41:38,767 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-33) [3ba86911] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-18 16:41:38,813 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-33) [754f0981] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-18 16:41:38,822 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-33) [754f0981] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = NFS, connectionList = [{ id: 1670ace3-7db6-4c73-9b3c-614dcc85bd46, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc5, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc4, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 6f4238e2 >2014-08-18 16:41:38,867 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-33) [754f0981] FINISH, ConnectStorageServerVDSCommand, return: {1670ace3-7db6-4c73-9b3c-614dcc85bd46=0, 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a=0}, log id: 6f4238e2 >2014-08-18 16:41:38,867 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-33) [754f0981] Host h1 storage connection was succeeded >2014-08-18 16:41:38,877 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-30) START, ConnectStoragePoolVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vdsId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, masterVersion = 1), log id: 40ce70ef >2014-08-18 16:41:44,416 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-30) FINISH, ConnectStoragePoolVDSCommand, log id: 40ce70ef >2014-08-18 16:41:44,431 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-33) [754f0981] START, SetMOMPolicyParametersVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: d1342c8 >2014-08-18 16:41:44,490 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-33) [754f0981] FINISH, SetMOMPolicyParametersVDSCommand, log id: d1342c8 >2014-08-18 16:41:44,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) [754f0981] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host h1 was set to Up. >2014-08-18 16:41:44,564 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) [754f0981] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h1. >2014-08-18 16:41:44,644 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-33) [79935533] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-18 16:43:29,212 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [7442d6a1] Lock Acquired to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 16:43:29,213 WARN [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [7442d6a1] CanDoAction of action CreateAllSnapshotsFromVm failed. Reasons:VAR__ACTION__CREATE,VAR__TYPE__SNAPSHOT,ACTION_TYPE_FAILED_IMAGE_REPOSITORY_NOT_FOUND >2014-08-18 16:43:29,214 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [7442d6a1] Lock freed to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 16:43:57,781 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (http--0.0.0.0-8080-3) [7ba289ae] Lock Acquired to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >, sharedLocks= ] >2014-08-18 16:43:57,862 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-3) [7ba289ae] Running command: ActivateStorageDomainCommand internal: false. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-08-18 16:43:57,878 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-3) [7ba289ae] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >, sharedLocks= ] >2014-08-18 16:43:57,880 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-3) [7ba289ae] ActivateStorage Domain. Before Connect all hosts to pool. Time:8/18/14 4:43 PM >2014-08-18 16:43:57,892 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-8-thread-2) [14241873] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-08-18 16:43:57,895 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-2) [14241873] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc4, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1ecdc09d >2014-08-18 16:43:57,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-2) [14241873] FINISH, ConnectStorageServerVDSCommand, return: {5ea693b3-ccb2-45a3-b82b-3cd6242cd79a=0}, log id: 1ecdc09d >2014-08-18 16:43:57,914 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-3) [7ba289ae] START, ActivateStorageDomainVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a), log id: 8951275 >2014-08-18 16:44:36,036 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-2) [7b5b7e93] IrsBroker::Failed::GetStoragePoolInfoVDS due to: TimeoutException: >2014-08-18 16:44:36,251 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-3) [7ba289ae] FINISH, ActivateStorageDomainVDSCommand, log id: 8951275 >2014-08-18 16:44:36,252 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-3) [7ba289ae] ActivateStorage Domain. After Connect all hosts to pool. Time:8/18/14 4:44 PM >2014-08-18 16:44:36,337 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-45) START, ConnectStoragePoolVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vdsId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, masterVersion = 1), log id: 3d028891 >2014-08-18 16:44:36,823 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-45) FINISH, ConnectStoragePoolVDSCommand, log id: 3d028891 >2014-08-18 16:44:36,824 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-3) [7ba289ae] ActivateStorage Domain. After change storage pool status in vds. Time:8/18/14 4:44 PM >2014-08-18 16:44:36,857 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-3) [7ba289ae] Correlation ID: 7ba289ae, Job ID: 847e4a2d-c6dc-44c4-a3fa-f8e05f554b85, Call Stack: null, Custom Event ID: -1, Message: Storage Domain sd_35 (Data Center Default) was activated by admin >2014-08-18 16:45:39,886 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [3598ad9d] Lock Acquired to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 16:45:39,898 ERROR [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [3598ad9d] Error during CanDoActionFailure.: java.lang.NullPointerException > at java.util.AbstractCollection.addAll(AbstractCollection.java:333) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.bll.ImagesHandler.getAllStorageIdsForImageIds(ImagesHandler.java:505) [bll.jar:] > at org.ovirt.engine.core.bll.VmCommand.createMultipleStorageDomainsValidator(VmCommand.java:556) [bll.jar:] > at org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.validateStorageDomains(CreateAllSnapshotsFromVmCommand.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.canDoAction(CreateAllSnapshotsFromVmCommand.java:524) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 16:45:39,925 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [3598ad9d] Lock freed to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 16:45:52,628 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [395c1e8a] Lock Acquired to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 16:47:08,398 ERROR [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [395c1e8a] Error during CanDoActionFailure.: java.lang.NullPointerException > at java.util.AbstractCollection.addAll(AbstractCollection.java:333) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.bll.ImagesHandler.getAllStorageIdsForImageIds(ImagesHandler.java:505) [bll.jar:] > at org.ovirt.engine.core.bll.VmCommand.createMultipleStorageDomainsValidator(VmCommand.java:556) [bll.jar:] > at org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.validateStorageDomains(CreateAllSnapshotsFromVmCommand.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.canDoAction(CreateAllSnapshotsFromVmCommand.java:524) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 16:47:08,438 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [395c1e8a] Lock freed to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 16:47:21,073 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [12992f9] Lock Acquired to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 16:50:30,909 ERROR [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [12992f9] Error during CanDoActionFailure.: java.lang.NullPointerException > at java.util.AbstractCollection.addAll(AbstractCollection.java:333) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.bll.ImagesHandler.getAllStorageIdsForImageIds(ImagesHandler.java:505) [bll.jar:] > at org.ovirt.engine.core.bll.VmCommand.createMultipleStorageDomainsValidator(VmCommand.java:556) [bll.jar:] > at org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.validateStorageDomains(CreateAllSnapshotsFromVmCommand.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.canDoAction(CreateAllSnapshotsFromVmCommand.java:524) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 16:50:30,980 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [12992f9] Lock freed to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 16:50:37,649 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [5f7d8afa] Lock Acquired to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 16:52:17,661 ERROR [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [5f7d8afa] Error during CanDoActionFailure.: java.lang.NullPointerException > at java.util.AbstractCollection.addAll(AbstractCollection.java:333) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.bll.ImagesHandler.getAllStorageIdsForImageIds(ImagesHandler.java:505) [bll.jar:] > at org.ovirt.engine.core.bll.VmCommand.createMultipleStorageDomainsValidator(VmCommand.java:556) [bll.jar:] > at org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.validateStorageDomains(CreateAllSnapshotsFromVmCommand.java:156) [bll.jar:] > at org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand.canDoAction(CreateAllSnapshotsFromVmCommand.java:524) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.internalCanDoAction(CommandBase.java:746) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.canDoActionOnly(CommandBase.java:321) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.execute(MultipleActionsRunner.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActionsImpl(Backend.java:547) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runMultipleActions(Backend.java:517) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.ovirt.engine.core.bll.interceptors.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:13) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view8.runMultipleActions(Unknown Source) [common.jar:] > at org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.runMultipleActions(GenericApiGWTServiceImpl.java:115) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196) > at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:172) > at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:233) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.HeaderFilter.doFilter(HeaderFilter.java:94) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.ui.frontend.server.gwt.GwtCachingFilter.doFilter(GwtCachingFilter.java:132) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.branding.BrandingFilter.doFilter(BrandingFilter.java:72) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.utils.servlet.LocaleFilter.doFilter(LocaleFilter.java:64) [utils.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionMgmtFilter.doFilter(SessionMgmtFilter.java:31) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.LoginFilter.doFilter(LoginFilter.java:73) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.NegotiationFilter.doFilter(NegotiationFilter.java:105) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:75) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:63) [aaa.jar:] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) > at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 16:52:17,709 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [5f7d8afa] Lock freed to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 17:00:48,551 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-18 17:00:48,552 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-18 17:00:48,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-18 17:00:48,558 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-18 17:00:48,560 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-18 17:00:48,561 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-18 17:00:48,561 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-18 17:00:48,562 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-18 17:00:48,562 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-18 17:00:48,563 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-18 17:00:48,563 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-18 17:00:48,564 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-18 17:00:48,564 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-18 17:00:48,565 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-18 17:00:48,565 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-18 17:00:48,566 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-18 17:00:48,566 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-18 17:00:48,567 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-18 17:00:48,567 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-18 17:00:48,568 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-18 17:00:48,568 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-18 17:00:48,569 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-18 17:00:48,570 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DB_USER" is "engine". >2014-08-18 17:00:48,571 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-18 17:00:48,571 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-18 17:00:48,572 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-18 17:00:48,572 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-18 17:00:48,573 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-18 17:00:48,574 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-18 17:00:48,574 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-18 17:00:48,575 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-18 17:00:48,575 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-18 17:00:48,575 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-18 17:00:48,576 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-18 17:00:48,577 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-18 17:00:48,577 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-18 17:00:48,577 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-18 17:00:48,578 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-18 17:00:48,579 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-18 17:00:48,579 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-18 17:00:48,580 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-18 17:00:48,580 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-18 17:00:48,581 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-18 17:00:48,581 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-18 17:00:48,582 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-18 17:00:48,582 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-18 17:00:48,583 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-18 17:00:48,584 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-18 17:00:48,584 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-18 17:00:48,585 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-18 17:00:48,585 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-18 17:00:48,586 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-18 17:00:48,586 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-18 17:00:48,587 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-18 17:00:48,588 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-18 17:00:48,588 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-18 17:00:48,589 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-18 17:00:48,589 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-18 17:00:48,590 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-18 17:00:48,590 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-18 17:00:48,591 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-18 17:00:48,591 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-18 17:00:48,592 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-18 17:00:48,593 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_USER" is "ovirt". >2014-08-18 17:00:48,594 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-18 17:00:48,594 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-18 17:00:48,595 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-18 17:00:48,596 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-18 17:00:48,596 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-18 17:00:48,597 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-2) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-18 17:00:48,985 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Start initializing Backend >2014-08-18 17:00:49,097 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 17:00:49,199 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-18 17:00:49,231 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-18 17:00:49,244 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-1) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-18 17:00:49,397 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Running ovirt-engine 3.5.0.0 >2014-08-18 17:00:49,397 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Start initializing dictionaries >2014-08-18 17:00:49,400 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Finished initializing dictionaries >2014-08-18 17:00:49,401 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing AuditLogCleanupManager >2014-08-18 17:00:49,401 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-18 17:00:49,414 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing AuditLogCleanupManager >2014-08-18 17:00:49,415 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing CommandEntityCleanupManager >2014-08-18 17:00:49,415 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-18 17:00:49,416 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing CommandEntityCleanupManager >2014-08-18 17:00:49,418 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Start initializing TagsDirector >2014-08-18 17:00:49,430 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Tag root added to tree >2014-08-18 17:00:49,438 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Finished initializing TagsDirector >2014-08-18 17:00:49,438 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Start initializing IsoDomainListSyncronizer >2014-08-18 17:00:49,442 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Finished initializing IsoDomainListSyncronizer >2014-08-18 17:00:49,521 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Completed initializing handlers >2014-08-18 17:00:49,543 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-08-18 17:00:49,553 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-18 17:00:49,554 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-08-18 17:00:49,554 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-08-18 17:00:49,555 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-08-18 17:00:49,555 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Mark incomplete jobs as UNKNOWN >2014-08-18 17:00:49,605 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Start initializing JobRepositoryCleanupManager >2014-08-18 17:00:49,606 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Finished initializing JobRepositoryCleanupManager >2014-08-18 17:00:49,606 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Start initializing AutoRecoveryManager >2014-08-18 17:00:49,607 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Finished initializing AutoRecoveryManager >2014-08-18 17:00:49,609 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Start initializing ExecutionMessageDirector >2014-08-18 17:00:49,611 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Finished initializing ExecutionMessageDirector >2014-08-18 17:00:49,650 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Start of enabled extensions list >2014-08-18 17:00:49,651 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-18 17:00:49,651 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-18 17:00:49,652 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) End of enabled extensions list >2014-08-18 17:00:49,705 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-3) Start initializing DbUserCacheManager >2014-08-18 17:00:49,707 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-3) Finished initializing DbUserCacheManager >2014-08-18 17:00:49,716 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-3) Initialization of AsyncTaskManager completed successfully. >2014-08-18 17:00:49,720 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-3) Start initializing ResourceManager >2014-08-18 17:00:49,741 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-08-18 17:00:49,793 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-3) Entered VdsManager constructor >2014-08-18 17:00:49,814 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-3) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-18 17:00:49,891 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-3) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-18 17:00:49,892 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-3) Entered VdsManager constructor >2014-08-18 17:00:49,904 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-3) Initialize vdsBroker (galactica,54,321) >2014-08-18 17:00:49,930 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-3) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-18 17:00:49,950 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-3) Finished initializing ResourceManager >2014-08-18 17:00:49,954 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-3) Initialization of OvfDataUpdater completed successfully. >2014-08-18 17:00:49,955 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) Start scheduling to enable vds load balancer >2014-08-18 17:00:49,955 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) Finished scheduling to enable vds load balancer >2014-08-18 17:00:49,955 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) Start HA Reservation check >2014-08-18 17:00:49,956 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) Finished HA Reservation check >2014-08-18 17:00:49,961 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-3) Init VM custom properties utilities >2014-08-18 17:00:49,962 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-3) Init device custom properties utilities >2014-08-18 17:00:49,966 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) Initializing Scheduling manager >2014-08-18 17:00:50,002 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-18 17:00:50,006 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-18 17:00:50,007 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-18 17:00:50,009 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) External scheduler disabled, discovery skipped >2014-08-18 17:00:50,010 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-3) Initialized Scheduling manager >2014-08-18 17:00:50,011 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-3) Initializing DWH Heart Beat >2014-08-18 17:00:50,015 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-3) DWH Heart Beat initialized >2014-08-18 17:00:52,945 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: h2 >2014-08-18 17:00:53,607 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 15690d77 >2014-08-18 17:00:53,615 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 15690d77 >2014-08-18 17:00:53,621 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Cannot convert the string Host ${VdsName} is not responding. It will stay in Connecting state for a grace period of ${Seconds} seconds and after that an attempt to fence the host will be issued. to AuditLogType, the key does not exist in the AuditLogType declared types >2014-08-18 17:00:53,625 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: UNASSIGNED not exist in string table >2014-08-18 17:00:53,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-18 17:00:53,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-18 17:00:53,627 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-18 17:00:53,627 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-18 17:00:53,628 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NOT_RESTARTED_DUE_TO_POLICY not exist in string table >2014-08-18 17:00:53,628 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-18 17:00:53,629 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-18 17:00:53,629 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-18 17:00:53,630 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HOST_IN_CONNECTING_STATE not exist in string table >2014-08-18 17:00:53,631 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-18 17:00:53,632 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STOPPED not exist in string table >2014-08-18 17:00:53,632 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STARTED not exist in string table >2014-08-18 17:00:53,633 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_ERROR not exist in string table >2014-08-18 17:00:53,633 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-08-18 17:00:53,634 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-08-18 17:00:53,634 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-08-18 17:00:53,635 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-08-18 17:00:53,635 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-08-18 17:00:53,636 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-08-18 17:00:53,637 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-18 17:00:53,637 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-18 17:00:53,638 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-18 17:00:53,639 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-18 17:00:53,781 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-18 17:00:53,782 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is running with disabled SELinux. >2014-08-18 17:00:54,274 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: h1 >2014-08-18 17:00:54,385 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [b5bb928] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-18 17:01:00,003 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-23) [251dcf2d] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-18 17:01:00,021 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [251dcf2d] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-18 17:01:00,034 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-23) [251dcf2d] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-18 17:01:00,166 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [251dcf2d] Correlation ID: 251dcf2d, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-08-18 17:01:00,302 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) [251dcf2d] hostFromVds::selectedVds - h1, spmStatus SPM, storage pool Default >2014-08-18 17:01:00,315 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) [251dcf2d] Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-18 17:01:00,340 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [251dcf2d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-18 17:01:00,361 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [251dcf2d] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false), log id: 1e495f0d >2014-08-18 17:01:01,434 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [251dcf2d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 27 GB of free space >2014-08-18 17:01:01,582 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-23) [251dcf2d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 27 GB of free space >2014-08-18 17:01:01,583 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [251dcf2d] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000037f >2014-08-18 17:01:01,590 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [251dcf2d] START, HSMGetAllTasksInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 47f2925c >2014-08-18 17:01:01,596 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [251dcf2d] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 47f2925c >2014-08-18 17:01:01,597 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [251dcf2d] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 1e495f0d >2014-08-18 17:01:01,597 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [251dcf2d] Discovered no tasks on Storage Pool Default >2014-08-18 17:01:21,338 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-4) Running command: LoginAdminUserCommand internal: false. >2014-08-18 17:01:21,377 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-18 17:01:24,240 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-5) Running command: LoginUserCommand internal: false. >2014-08-18 17:02:06,516 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-4) [76367bbb] Lock Acquired to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 17:02:06,601 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-36) Command b84aaf3e-a376-4cfa-b042-c50fecd4af9e persisting async task placeholder for child command a4ae318b-9a75-48ca-8600-5e4ba05d283e >2014-08-18 17:02:06,747 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-36) Running command: CreateAllSnapshotsFromVmCommand internal: false. Entities affected : ID: 337241af-a081-4971-a3d2-4c7a790e030d Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER >2014-08-18 17:02:06,888 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-36) [b8ee640] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-08-18 17:02:06,897 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-36) [b8ee640] START, CreateSnapshotVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 2b033ab4-8cd6-4214-baf6-1252264f371e, imageSizeInBytes = 10737418240, volumeFormat = COW, newImageId = dffc37f7-bbfc-4d95-8be7-a31be5899391, newImageDescription = , imageId = dcdd4d3f-02ff-4494-8987-5a5f633927d1, sourceImageGroupId = 2b033ab4-8cd6-4214-baf6-1252264f371e), log id: 13504038 >2014-08-18 17:02:06,903 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-36) [b8ee640] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-18 17:02:07,658 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-36) [b8ee640] FINISH, CreateSnapshotVDSCommand, return: dffc37f7-bbfc-4d95-8be7-a31be5899391, log id: 13504038 >2014-08-18 17:02:07,731 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-36) [b8ee640] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command b84aaf3e-a376-4cfa-b042-c50fecd4af9e >2014-08-18 17:02:07,732 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-36) [b8ee640] CommandMultiAsyncTasks::AttachTask: Attaching task 8cb99227-6ff3-4854-836e-35de48369e9e to command b84aaf3e-a376-4cfa-b042-c50fecd4af9e. >2014-08-18 17:02:07,758 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-36) [b8ee640] Adding task 8cb99227-6ff3-4854-836e-35de48369e9e (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-18 17:02:07,965 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-36) Correlation ID: 76367bbb, Job ID: 10f3b758-6a86-493c-bab3-eb99b608d11f, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'dd' creation for VM 'vm2' was initiated by admin. >2014-08-18 17:02:07,966 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-36) BaseAsyncTask::startPollingTask: Starting to poll task 8cb99227-6ff3-4854-836e-35de48369e9e. >2014-08-18 17:02:09,715 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-11) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-18 17:02:09,773 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-11) SPMAsyncTask::PollTask: Polling task 8cb99227-6ff3-4854-836e-35de48369e9e (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-08-18 17:02:09,775 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-11) Finished polling Tasks, will poll again in 10 seconds. >2014-08-18 17:02:19,848 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-39) SPMAsyncTask::PollTask: Polling task 8cb99227-6ff3-4854-836e-35de48369e9e (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-18 17:02:19,883 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-39) BaseAsyncTask::onTaskEndSuccess: Task 8cb99227-6ff3-4854-836e-35de48369e9e (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-18 17:02:19,884 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-39) CommandAsyncTask::endActionIfNecessary: All tasks of command b84aaf3e-a376-4cfa-b042-c50fecd4af9e has ended -> executing endAction >2014-08-18 17:02:19,885 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-39) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: b84aaf3e-a376-4cfa-b042-c50fecd4af9e): calling endAction . >2014-08-18 17:02:19,887 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-3) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction CreateAllSnapshotsFromVm, executionIndex: 0 >2014-08-18 17:02:19,941 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-3) Ending command successfully: org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand >2014-08-18 17:02:19,968 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-3) [b8ee640] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotCommand >2014-08-18 17:02:19,977 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-3) [b8ee640] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 2b033ab4-8cd6-4214-baf6-1252264f371e, imageId = dffc37f7-bbfc-4d95-8be7-a31be5899391), log id: 64d4dcbe >2014-08-18 17:02:20,004 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-3) [b8ee640] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@a5573b49, log id: 64d4dcbe >2014-08-18 17:02:20,054 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-3) Lock freed to object EngineLock [exclusiveLocks= key: 337241af-a081-4971-a3d2-4c7a790e030d value: VM >, sharedLocks= ] >2014-08-18 17:02:20,082 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-3) Correlation ID: 76367bbb, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'dd' creation for VM 'vm2' has been completed. >2014-08-18 17:02:20,083 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-3) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type CreateAllSnapshotsFromVm completed, handling the result. >2014-08-18 17:02:20,083 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-3) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type CreateAllSnapshotsFromVm succeeded, clearing tasks. >2014-08-18 17:02:20,099 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-3) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 8cb99227-6ff3-4854-836e-35de48369e9e >2014-08-18 17:02:20,100 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-3) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 8cb99227-6ff3-4854-836e-35de48369e9e), log id: 3bfe185c >2014-08-18 17:02:20,103 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-3) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=8cb99227-6ff3-4854-836e-35de48369e9e), log id: 4ff1d5cb >2014-08-18 17:02:22,250 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-3) FINISH, HSMClearTaskVDSCommand, log id: 4ff1d5cb >2014-08-18 17:02:22,251 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-3) FINISH, SPMClearTaskVDSCommand, log id: 3bfe185c >2014-08-18 17:02:22,277 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-3) BaseAsyncTask::removeTaskFromDB: Removed task 8cb99227-6ff3-4854-836e-35de48369e9e from DataBase >2014-08-18 17:02:22,278 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-3) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity b84aaf3e-a376-4cfa-b042-c50fecd4af9e >2014-08-18 17:03:49,710 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-85) Setting new tasks map. The map contains now 0 tasks >2014-08-18 17:03:49,711 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-85) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-18 17:26:42,375 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) Command SpmStatusVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-18 17:26:42,379 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-17) [4c8b1e4c] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-18 17:26:42,383 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-17) [4c8b1e4c] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-18 17:26:42,386 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-17) [4c8b1e4c] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-18 17:26:42,440 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) [4c8b1e4c] Correlation ID: 4c8b1e4c, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host h1, Error: Network error during communication with the Host.). >2014-08-28 11:34:01,514 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-08-28 11:34:01,548 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) The file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-08-28 11:34:01,550 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-08-28 11:34:01,551 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-08-28 11:34:01,552 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-08-28 11:34:01,552 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Loaded file "/home/vvolansk/ovirt-engine/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-08-28 11:34:01,553 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_AJP_ENABLED" is "false". >2014-08-28 11:34:01,553 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_AJP_PORT" is "None". >2014-08-28 11:34:01,554 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war". >2014-08-28 11:34:01,554 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_CACHE" is "/home/vvolansk/ovirt-engine/var/cache/ovirt-engine". >2014-08-28 11:34:01,554 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-08-28 11:34:01,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-08-28 11:34:01,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-08-28 11:34:01,556 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-08-28 11:34:01,556 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_HOST" is "localhost". >2014-08-28 11:34:01,557 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-08-28 11:34:01,557 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-08-28 11:34:01,558 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-08-28 11:34:01,558 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_PORT" is "5432". >2014-08-28 11:34:01,559 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_SECURED" is "False". >2014-08-28 11:34:01,559 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-08-28 11:34:01,559 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-08-28 11:34:01,560 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DB_USER" is "engine". >2014-08-28 11:34:01,560 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DEBUG_ADDRESS" is "127.0.0.1:8787". >2014-08-28 11:34:01,561 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_DOC" is "/home/vvolansk/ovirt-engine/share/doc/ovirt-engine". >2014-08-28 11:34:01,562 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_ETC" is "/home/vvolansk/ovirt-engine/etc/ovirt-engine". >2014-08-28 11:34:01,562 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_EXTENSION_PATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/extensions.d:/home/vvolansk/ovirt-engine/etc/ovirt-engine/extensions.d". >2014-08-28 11:34:01,563 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_FQDN" is "vered.laptop.tlv.redhat.com". >2014-08-28 11:34:01,564 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_GROUP" is "ovirt". >2014-08-28 11:34:01,565 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-08-28 11:34:01,565 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-08-28 11:34:01,565 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTPS_ENABLED" is "true". >2014-08-28 11:34:01,566 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTPS_PORT" is "8443". >2014-08-28 11:34:01,568 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-08-28 11:34:01,568 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTP_ENABLED" is "true". >2014-08-28 11:34:01,569 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_HTTP_PORT" is "8080". >2014-08-28 11:34:01,569 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_JAVA_MODULEPATH" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/modules". >2014-08-28 11:34:01,569 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/home/vvolansk/ovirt-engine/var/log/ovirt-engine/dump"". >2014-08-28 11:34:01,570 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_LOG" is "/home/vvolansk/ovirt-engine/var/log/ovirt-engine". >2014-08-28 11:34:01,570 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_LOG_TO_CONSOLE" is "true". >2014-08-28 11:34:01,571 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_MANUAL" is "/home/vvolansk/ovirt-engine/share/ovirt-engine/manual". >2014-08-28 11:34:01,571 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PERM_MAX" is "256m". >2014-08-28 11:34:01,572 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PERM_MIN" is "256m". >2014-08-28 11:34:01,572 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine". >2014-08-28 11:34:01,573 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_CA" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/ca.pem". >2014-08-28 11:34:01,573 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_CERT" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/certs/engine.cer". >2014-08-28 11:34:01,574 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/keys/engine.p12". >2014-08-28 11:34:01,574 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-08-28 11:34:01,575 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-08-28 11:34:01,575 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_ENGINE_STORE_TYPE" is "PKCS12". >2014-08-28 11:34:01,576 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_TRUST_STORE" is "/home/vvolansk/ovirt-engine/etc/pki/ovirt-engine/.truststore". >2014-08-28 11:34:01,577 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-08-28 11:34:01,577 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PKI_TRUST_STORE_TYPE" is "JKS". >2014-08-28 11:34:01,578 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/home/vvolansk/ovirt-engine/etc/ovirt-engine/krb5.conf"". >2014-08-28 11:34:01,578 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROXY_ENABLED" is "false". >2014-08-28 11:34:01,579 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-08-28 11:34:01,579 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-08-28 11:34:01,579 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_REPORTS_UI" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/reports.xml". >2014-08-28 11:34:01,580 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-08-28 11:34:01,580 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_STOP_TIME" is "10". >2014-08-28 11:34:01,581 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_TMP" is "/home/vvolansk/ovirt-engine/var/tmp/ovirt-engine". >2014-08-28 11:34:01,581 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_UP_MARK" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/engine.up". >2014-08-28 11:34:01,582 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-08-28 11:34:01,582 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_USER" is "ovirt". >2014-08-28 11:34:01,582 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_USR" is "/home/vvolansk/ovirt-engine/share/ovirt-engine". >2014-08-28 11:34:01,583 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_VAR" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine". >2014-08-28 11:34:01,583 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-08-28 11:34:01,583 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-08-28 11:34:01,584 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "JBOSS_RUNTIME" is "/home/vvolansk/ovirt-engine/var/lib/ovirt-engine/jboss_runtime". >2014-08-28 11:34:01,584 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (MSC service thread 1-6) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-08-28 11:34:01,859 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Start initializing Backend >2014-08-28 11:34:01,992 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Failed to decrypt value for property AdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:34:02,081 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Failed to decrypt value for property LocalAdminPassword will be used encrypted value: javax.crypto.BadPaddingException: Data must start with zero > at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:325) [rt.jar:1.7.0_25] > at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:272) [rt.jar:1.7.0_25] > at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:356) > at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:382) > at javax.crypto.Cipher.doFinal(Cipher.java:1922) [jce.jar:1.7.0_25] > at org.ovirt.engine.core.utils.crypt.EngineEncryptionUtils.decrypt(EngineEncryptionUtils.java:163) [utils.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.getValue(DBConfigUtils.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.updateOption(DBConfigUtils.java:263) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.refreshVdcOptionCache(DBConfigUtils.java:38) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:171) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils.<init>(DBConfigUtils.java:157) [dal.jar:] > at org.ovirt.engine.core.bll.Backend.initialize(Backend.java:187) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.create(Backend.java:138) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:130) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:73) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ManagedReferenceInterceptorFactory$ManagedReferenceInterceptor.processInvocation(ManagedReferenceInterceptorFactory.java:95) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:333) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.SingletonLifecycleCMTTxInterceptor.processInvocation(SingletonLifecycleCMTTxInterceptor.java:56) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:85) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:116) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:130) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ComponentStartService.start(ComponentStartService.java:44) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:34:02,116 ERROR [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (MSC service thread 1-1) Could not parse option AutoRecoveryAllowedTypes value. >2014-08-28 11:34:03,843 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-1) Loaded file /home/vvolansk/ovirt-engine/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-08-28 11:34:04,301 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Running ovirt-engine 3.5.0.0 >2014-08-28 11:34:04,302 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Start initializing dictionaries >2014-08-28 11:34:04,306 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Finished initializing dictionaries >2014-08-28 11:34:04,306 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing AuditLogCleanupManager >2014-08-28 11:34:04,307 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-08-28 11:34:04,327 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing AuditLogCleanupManager >2014-08-28 11:34:04,327 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing CommandEntityCleanupManager >2014-08-28 11:34:04,327 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-08-28 11:34:04,329 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing CommandEntityCleanupManager >2014-08-28 11:34:04,331 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Start initializing TagsDirector >2014-08-28 11:34:04,339 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Tag root added to tree >2014-08-28 11:34:04,348 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Finished initializing TagsDirector >2014-08-28 11:34:04,348 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Start initializing IsoDomainListSyncronizer >2014-08-28 11:34:04,355 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Finished initializing IsoDomainListSyncronizer >2014-08-28 11:34:04,429 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Completed initializing handlers >2014-08-28 11:34:04,449 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-08-28 11:34:04,460 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-08-28 11:34:04,461 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-08-28 11:34:04,462 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-08-28 11:34:04,463 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-08-28 11:34:04,463 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Mark incomplete jobs as UNKNOWN >2014-08-28 11:34:04,539 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Start initializing JobRepositoryCleanupManager >2014-08-28 11:34:04,539 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Finished initializing JobRepositoryCleanupManager >2014-08-28 11:34:04,540 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Start initializing AutoRecoveryManager >2014-08-28 11:34:04,541 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Finished initializing AutoRecoveryManager >2014-08-28 11:34:04,542 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Start initializing ExecutionMessageDirector >2014-08-28 11:34:04,544 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Finished initializing ExecutionMessageDirector >2014-08-28 11:34:04,610 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Loading extension 'builtin-authn-internal' >2014-08-28 11:34:04,611 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Extension 'builtin-authn-internal' loaded >2014-08-28 11:34:04,613 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Loading extension 'internal' >2014-08-28 11:34:04,614 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Extension 'internal' loaded >2014-08-28 11:34:04,629 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Initializing extension 'builtin-authn-internal' >2014-08-28 11:34:04,630 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Extension 'builtin-authn-internal' initialized >2014-08-28 11:34:04,631 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Initializing extension 'internal' >2014-08-28 11:34:04,631 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Extension 'internal' initialized >2014-08-28 11:34:04,631 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Start of enabled extensions list >2014-08-28 11:34:04,631 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-28 11:34:04,632 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-08-28 11:34:04,633 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (MSC service thread 1-1) End of enabled extensions list >2014-08-28 11:34:04,737 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-2) Start initializing DbUserCacheManager >2014-08-28 11:34:04,738 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (MSC service thread 1-2) Finished initializing DbUserCacheManager >2014-08-28 11:34:04,753 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (MSC service thread 1-2) Initialization of AsyncTaskManager completed successfully. >2014-08-28 11:34:04,754 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-2) Start initializing ResourceManager >2014-08-28 11:34:04,840 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-2) Entered VdsManager constructor >2014-08-28 11:34:04,881 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-2) Initialize vdsBroker (battlestar.tlv.redhat.com,54,321) >2014-08-28 11:34:05,003 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-2) VDS 9c67d903-8424-4cda-8bb1-3d8652b92d77 was added to the Resource Manager >2014-08-28 11:34:05,004 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-2) Entered VdsManager constructor >2014-08-28 11:34:05,016 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-2) Initialize vdsBroker (galactica,54,321) >2014-08-28 11:34:05,039 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-2) VDS eb4134dd-44f4-4ce1-a9c4-c1054a82b7e6 was added to the Resource Manager >2014-08-28 11:34:05,056 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-2) Finished initializing ResourceManager >2014-08-28 11:34:05,060 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-2) Initialization of OvfDataUpdater completed successfully. >2014-08-28 11:34:05,061 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Start scheduling to enable vds load balancer >2014-08-28 11:34:05,061 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Finished scheduling to enable vds load balancer >2014-08-28 11:34:05,062 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Start HA Reservation check >2014-08-28 11:34:05,063 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Finished HA Reservation check >2014-08-28 11:34:05,071 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-2) Init VM custom properties utilities >2014-08-28 11:34:05,072 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-2) Init device custom properties utilities >2014-08-28 11:34:05,078 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Initializing Scheduling manager >2014-08-28 11:34:05,124 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Start initializing MacPoolManagerRanges >2014-08-28 11:34:05,124 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolManagerRanges] (org.ovirt.thread.pool-8-thread-1) Finished initializing. Available MACs in pool: 150 >2014-08-28 11:34:05,125 INFO [org.ovirt.engine.core.bll.network.macpoolmanager.MacPoolPerDc] (org.ovirt.thread.pool-8-thread-1) successfully initialized >2014-08-28 11:34:05,132 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) External scheduler disabled, discovery skipped >2014-08-28 11:34:05,132 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-2) Initialized Scheduling manager >2014-08-28 11:34:05,133 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-2) Initializing DWH Heart Beat >2014-08-28 11:34:05,134 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-2) DWH Heart Beat initialized >2014-08-28 11:34:08,126 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: h2 >2014-08-28 11:34:09,948 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (http--0.0.0.0-8080-1) Invalid locale found in configuration: >2014-08-28 11:34:09,948 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (http--0.0.0.0-8080-1) Invalid locale found in configuration: >2014-08-28 11:34:10,543 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetStatsVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:10,551 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 81 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:34:10,595 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: UNASSIGNED not exist in string table >2014-08-28 11:34:10,597 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-08-28 11:34:10,597 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-08-28 11:34:10,598 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-08-28 11:34:10,599 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-08-28 11:34:10,599 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-08-28 11:34:10,600 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-08-28 11:34:10,601 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-08-28 11:34:10,602 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-08-28 11:34:10,603 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STOPPED not exist in string table >2014-08-28 11:34:10,603 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_STARTED not exist in string table >2014-08-28 11:34:10,604 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: DWH_ERROR not exist in string table >2014-08-28 11:34:10,604 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-08-28 11:34:10,605 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-08-28 11:34:10,606 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-08-28 11:34:10,606 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-08-28 11:34:10,607 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-08-28 11:34:10,607 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-08-28 11:34:10,608 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-08-28 11:34:10,608 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-08-28 11:34:10,609 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-08-28 11:34:10,609 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-08-28 11:34:10,610 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-08-28 11:34:10,610 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-08-28 11:34:10,611 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-08-28 11:34:10,612 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-08-28 11:34:10,612 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-08-28 11:34:10,613 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-08-28 11:34:11,338 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 81 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:34:11,339 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) vds::refreshVdsStats Failed getVdsStats, vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, error = VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:11,340 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsStats(VdsUpdateRunTimeInfo.java:672) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:626) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsStats(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:34:11,418 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:15,884 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (http--0.0.0.0-8080-2) Invalid locale found in configuration: >2014-08-28 11:34:15,885 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (http--0.0.0.0-8080-2) Invalid locale found in configuration: >2014-08-28 11:34:16,463 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:16,464 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 81 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:34:16,489 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:34:16,521 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:21,610 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:21,612 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 81 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:34:21,691 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (DefaultQuartzScheduler_Worker-31) START, SetVmStatusVDSCommand( vmId = de580b19-e975-426f-80de-6c7daf1f24ab, status = Unknown, exit status = Normal), log id: 290b158d >2014-08-28 11:34:21,714 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, SetVmStatusVDSCommand, log id: 290b158d >2014-08-28 11:34:21,846 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-31) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm4 was set to the Unknown status. >2014-08-28 11:34:21,847 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (DefaultQuartzScheduler_Worker-31) START, SetVmStatusVDSCommand( vmId = 5b1ee280-4251-489a-899c-8e9dac73c327, status = Unknown, exit status = Normal), log id: 6e548a6a >2014-08-28 11:34:21,854 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, SetVmStatusVDSCommand, log id: 6e548a6a >2014-08-28 11:34:21,896 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-31) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM vm3 was set to the Unknown status. >2014-08-28 11:34:21,896 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Server failed to respond, vds_id = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds_name = h1, vm_count = 2, spm_status = SPM, non-responsive_timeout (seconds) = 81, error = java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:21,912 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-31) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) >, Custom Event ID: -1, Message: Host h1 is non responsive. >2014-08-28 11:34:21,933 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-8-thread-7) ResourceManager::vdsNotResponding entered for Host 9c67d903-8424-4cda-8bb1-3d8652b92d77, battlestar.tlv.redhat.com >2014-08-28 11:34:21,979 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:34:22,235 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-7) [6614e894] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-28 11:34:22,248 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-7) [6614e894] Opening SSH Soft Fencing session on host battlestar.tlv.redhat.com >2014-08-28 11:34:22,616 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-7) [6614e894] SSH Soft Fencing command failed on host battlestar.tlv.redhat.com: Failed to get the session. >Stdout: >Stderr: >Stacktrace: org.apache.sshd.common.RuntimeSshException: Failed to get the session.: org.apache.sshd.common.RuntimeSshException: Failed to get the session. > at org.apache.sshd.client.future.DefaultConnectFuture.getSession(DefaultConnectFuture.java:43) > at org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:294) [uutils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:87) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:53) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1169) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1300) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1922) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1321) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:346) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view8.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$4.run(VdsEventListener.java:243) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishConnect(UnixAsynchronousSocketChannelImpl.java:252) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:213) [rt.jar:1.7.0_25] > at sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:293) [rt.jar:1.7.0_25] > ... 1 more > >2014-08-28 11:34:22,759 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-7) [14c89ce8] Lock Acquired to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-28 11:34:22,760 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-7) [14c89ce8] Fence failed on vds h1 which is spm of pool 00000002-0002-0002-0002-00000000037f - moving pool to non operational >2014-08-28 11:34:22,768 WARN [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (org.ovirt.thread.pool-8-thread-7) [b5e7241] The message key SetStoragePoolStatus is missing from bundles/ExecutionMessages >2014-08-28 11:34:22,815 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-8-thread-7) [b5e7241] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-28 11:34:22,898 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-7) [b5e7241] Correlation ID: b5e7241, Job ID: 3a960ee8-a205-4d57-bbae-36841d0b6178, Call Stack: null, Custom Event ID: -1, Message: Fencing failed on Storage Pool Manager h1 for Data Center Default. Setting status to Non-Operational. >2014-08-28 11:34:22,906 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-8-thread-7) [b5e7241] Lock freed to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-28 11:34:22,907 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-7) [b5e7241] Failed to run Fence script on vds:h1. >2014-08-28 11:34:23,113 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-7) [b5e7241] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-08-28 11:34:23,114 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-7) [b5e7241] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-08-28 11:34:23,114 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-7) [b5e7241] Trying to release exclusive lock which does not exist, lock key: 9c67d903-8424-4cda-8bb1-3d8652b92d77VDS_FENCE >2014-08-28 11:34:23,115 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-7) [b5e7241] Lock freed to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-28 11:34:27,054 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:27,056 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:34:27,098 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:34:32,142 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:32,144 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:34:32,422 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:34:33,909 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (http--0.0.0.0-8080-2) Running command: LoginAdminUserCommand internal: false. >2014-08-28 11:34:33,970 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-08-28 11:34:37,511 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:37,513 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:34:37,572 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:34:37,993 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (http--0.0.0.0-8080-5) Running command: LoginUserCommand internal: false. >2014-08-28 11:34:42,654 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:42,655 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:34:42,723 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:34:47,782 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:47,782 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:34:47,874 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:34:52,945 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:52,947 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:34:53,051 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:34:58,076 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:34:58,078 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:34:58,195 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:35:03,279 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:35:03,280 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:35:03,446 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:35:08,468 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:35:08,503 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:35:08,978 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:35:13,657 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (http--0.0.0.0-8080-2) [6903ac8b] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: POOL >] >2014-08-28 11:35:13,687 WARN [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (http--0.0.0.0-8080-2) [6903ac8b] CanDoAction of action MaintenanceNumberOfVdss failed. Reasons:VAR__TYPE__HOST,VAR__ACTION__MAINTENANCE,VDS_CANNOT_MAINTENANCE_VDS_IS_NOT_RESPONDING_WITH_VMS,$HostNotResponding h1 >2014-08-28 11:35:13,688 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (http--0.0.0.0-8080-2) [6903ac8b] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: POOL >] >2014-08-28 11:35:14,059 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:35:14,060 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:35:14,088 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:35:19,114 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:35:19,115 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:35:19,199 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:35:24,220 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:35:24,220 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:35:24,276 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:35:29,301 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:35:29,303 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:35:29,362 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:35:34,384 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:35:34,385 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:35:34,474 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:35:39,493 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:35:39,495 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:35:39,567 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:35:44,391 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-28 11:35:44,393 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:35:44,428 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:35:49,453 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:35:49,454 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:35:49,572 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:35:54,611 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:35:54,613 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:35:54,775 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:35:59,814 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:35:59,815 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:35:59,886 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:36:04,923 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:36:04,925 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:36:05,030 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:36:10,057 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:36:10,059 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:36:10,256 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:36:14,361 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-28 11:36:14,363 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:36:14,423 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:36:19,521 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:36:19,523 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:36:19,583 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:36:24,631 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:36:24,633 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:36:24,761 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:36:29,845 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:36:29,847 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:36:29,889 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:36:34,965 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:36:34,967 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:36:35,049 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:36:40,097 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:36:40,099 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:36:40,226 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:36:45,269 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:36:45,271 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:36:45,345 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:36:50,429 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:36:50,431 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:36:50,547 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:36:55,589 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:36:55,590 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:36:55,691 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:37:00,723 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:37:00,763 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:37:00,861 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:37:05,902 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:37:05,904 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:37:05,980 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:37:11,020 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:37:11,022 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:37:11,116 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-84) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:37:11,174 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:37:15,867 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-28 11:37:15,869 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:37:15,936 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:37:19,869 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-28 11:37:19,871 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:37:19,929 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:37:24,967 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-28 11:37:24,969 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:37:25,081 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:37:29,307 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-28 11:37:29,309 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host h1 is not responding. It will stay in Connecting state for a grace period of 162 seconds and after that an attempt to fence the host will be issued. >2014-08-28 11:37:29,372 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-28 11:37:33,385 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-42) START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 4213042b >2014-08-28 11:37:33,482 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-42) FINISH, GetHardwareInfoVDSCommand, log id: 4213042b >2014-08-28 11:37:33,505 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-42) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 does not enforce SELinux. >2014-08-28 11:37:33,506 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) Host h1 is running with disabled SELinux. >2014-08-28 11:37:33,584 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-42) [2a7222aa] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-28 11:37:33,822 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-42) [2a7222aa] START, GetHardwareInfoVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]), log id: 1d1c96a2 >2014-08-28 11:37:33,831 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-42) [2a7222aa] FINISH, GetHardwareInfoVDSCommand, log id: 1d1c96a2 >2014-08-28 11:37:33,836 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [2a7222aa] Host h1 is running with disabled SELinux. >2014-08-28 11:37:33,875 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [2a7222aa] VM vm4 (de580b19-e975-426f-80de-6c7daf1f24ab) is running in db and not running in VDS h1 >2014-08-28 11:37:33,876 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [2a7222aa] VM vm3 (5b1ee280-4251-489a-899c-8e9dac73c327) is running in db and not running in VDS h1 >2014-08-28 11:37:33,882 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-42) [4d44f807] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-28 11:37:33,908 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-42) [3bd2d787] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-28 11:37:33,961 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-42) [3bd2d787] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, storageType = NFS, connectionList = [{ id: 1670ace3-7db6-4c73-9b3c-614dcc85bd46, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc5, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc4, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 5664664d >2014-08-28 11:37:35,251 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-42) [3bd2d787] FINISH, ConnectStorageServerVDSCommand, return: {1670ace3-7db6-4c73-9b3c-614dcc85bd46=0, 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a=0}, log id: 5664664d >2014-08-28 11:37:35,254 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-42) [3bd2d787] Host h1 storage connection was succeeded >2014-08-28 11:37:35,276 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-43) START, ConnectStoragePoolVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vdsId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, masterVersion = 1), log id: 76f2da4e >2014-08-28 11:37:35,650 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-43) FINISH, ConnectStoragePoolVDSCommand, log id: 76f2da4e >2014-08-28 11:37:35,950 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-42) [3bd2d787] START, SetMOMPolicyParametersVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 636c9eaf >2014-08-28 11:37:36,003 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-42) [3bd2d787] FINISH, SetMOMPolicyParametersVDSCommand, log id: 636c9eaf >2014-08-28 11:37:36,047 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-42) [3bd2d787] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host h1 was set to Up. >2014-08-28 11:37:36,055 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-42) [3bd2d787] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host h1. >2014-08-28 11:37:36,165 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [3bd2d787] Initializing Host: h1 >2014-08-28 11:37:36,168 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-28 11:37:36,208 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-49) [109c4774] Running command: ProcessDownVmCommand internal: true. >2014-08-28 11:37:36,210 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-48) [41ee2964] Running command: ProcessDownVmCommand internal: true. >2014-08-28 11:37:36,274 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-49) [7161a5c2] Running command: ProcessDownVmCommand internal: true. >2014-08-28 11:37:36,286 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-8-thread-48) [7f4e7cb] Running command: ProcessDownVmCommand internal: true. >2014-08-28 11:37:45,514 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-80) hostFromVds::selectedVds - h1, spmStatus Free, storage pool Default >2014-08-28 11:37:45,670 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-80) SpmStatus on vds 9c67d903-8424-4cda-8bb1-3d8652b92d77: Free >2014-08-28 11:37:45,697 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-80) starting spm on vds h1, storage pool Default, prevId 1, LVER 4 >2014-08-28 11:37:45,702 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-80) START, SpmStartVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, prevId=1, prevLVER=4, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 43044309 >2014-08-28 11:37:45,713 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-80) spmStart polling started: taskId = 3ce7c921-d2bc-4d59-841f-b871085bc440 >2014-08-28 11:39:53,886 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (http--0.0.0.0-8080-2) [46659f66] Lock Acquired to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >, sharedLocks= ] >2014-08-28 11:39:54,209 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-29) [46659f66] Running command: ActivateStorageDomainCommand internal: false. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-08-28 11:39:54,300 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-29) [46659f66] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >, sharedLocks= ] >2014-08-28 11:39:54,301 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-29) [46659f66] ActivateStorage Domain. Before Connect all hosts to pool. Time:8/28/14 11:39 AM >2014-08-28 11:39:54,356 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-8-thread-32) [63e12b4d] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-08-28 11:39:54,359 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-32) [63e12b4d] START, ConnectStorageServerVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 5ea693b3-ccb2-45a3-b82b-3cd6242cd79a, connection: multipass.eng.lab.tlv.redhat.com:/export/images/rnd/vered/abc4, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 19c9c852 >2014-08-28 11:39:54,383 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-32) [63e12b4d] FINISH, ConnectStorageServerVDSCommand, return: {5ea693b3-ccb2-45a3-b82b-3cd6242cd79a=0}, log id: 19c9c852 >2014-08-28 11:39:54,385 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-29) [46659f66] START, ActivateStorageDomainVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a), log id: 50be8e1c >2014-08-28 11:40:17,686 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-80) spmStart polling ended: taskId = 3ce7c921-d2bc-4d59-841f-b871085bc440 task status = finished >2014-08-28 11:40:17,735 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-80) spmStart polling ended, spm status: SPM >2014-08-28 11:40:17,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-80) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=3ce7c921-d2bc-4d59-841f-b871085bc440), log id: 62b7fe4d >2014-08-28 11:40:17,768 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-80) FINISH, HSMClearTaskVDSCommand, log id: 62b7fe4d >2014-08-28 11:40:17,769 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-80) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@6644924, log id: 43044309 >2014-08-28 11:40:17,809 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-80) Initialize Irs proxy from vds: battlestar.tlv.redhat.com >2014-08-28 11:40:17,825 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host h1 (Address: battlestar.tlv.redhat.com). >2014-08-28 11:40:18,115 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd_35 domain has 17 GB of free space >2014-08-28 11:40:18,148 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Warning, Low disk space.sd35_1 domain has 17 GB of free space >2014-08-28 11:40:18,158 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-29) [46659f66] FINISH, ActivateStorageDomainVDSCommand, log id: 50be8e1c >2014-08-28 11:40:18,159 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-29) [46659f66] ActivateStorage Domain. After Connect all hosts to pool. Time:8/28/14 11:40 AM >2014-08-28 11:40:18,196 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-31) START, ConnectStoragePoolVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vdsId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f, masterVersion = 1), log id: 563db1ef >2014-08-28 11:40:18,302 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-31) FINISH, ConnectStoragePoolVDSCommand, log id: 563db1ef >2014-08-28 11:40:18,303 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-29) [46659f66] ActivateStorage Domain. After change storage pool status in vds. Time:8/28/14 11:40 AM >2014-08-28 11:40:18,339 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-29) [46659f66] Correlation ID: 46659f66, Job ID: 9f9539ce-70d1-44fd-8dba-d18f80ffebfb, Call Stack: null, Custom Event ID: -1, Message: Storage Domain sd_35 (Data Center Default) was activated by admin >2014-08-28 12:33:12,762 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (http--0.0.0.0-8080-2) [63cd7894] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-28 12:33:12,808 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (http--0.0.0.0-8080-2) [63cd7894] Running command: AddDiskCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group CONFIGURE_VM_STORAGE with role type USER, ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: StorageAction group CREATE_DISK with role type USER >2014-08-28 12:33:12,949 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (http--0.0.0.0-8080-2) [101f8991] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-28 12:33:12,999 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (http--0.0.0.0-8080-2) [101f8991] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >] >2014-08-28 12:33:13,001 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (http--0.0.0.0-8080-2) [101f8991] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 5431d17c-df3f-420c-b516-031e4ca4cf0a, imageSizeInBytes = 1073741824, volumeFormat = RAW, newImageId = 7123c19f-6adc-4690-9bf0-c26078190321, newImageDescription = ), log id: 52127477 >2014-08-28 12:33:13,005 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (http--0.0.0.0-8080-2) [101f8991] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-28 12:33:13,448 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (http--0.0.0.0-8080-2) [101f8991] FINISH, CreateImageVDSCommand, return: 7123c19f-6adc-4690-9bf0-c26078190321, log id: 52127477 >2014-08-28 12:33:13,586 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (http--0.0.0.0-8080-2) [101f8991] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command e9e9397a-b09f-45c9-bbb7-587d53e441fd >2014-08-28 12:33:13,587 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (http--0.0.0.0-8080-2) [101f8991] CommandMultiAsyncTasks::AttachTask: Attaching task 5e25b889-c7d0-4546-b615-1ff290d1d78b to command e9e9397a-b09f-45c9-bbb7-587d53e441fd. >2014-08-28 12:33:13,645 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (http--0.0.0.0-8080-2) [101f8991] Adding task 5e25b889-c7d0-4546-b615-1ff290d1d78b (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-28 12:33:14,168 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [101f8991] Correlation ID: 63cd7894, Job ID: 99a1e7e5-fa4e-4268-bda6-b1e017c74d2a, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of vm3_Disk3 was initiated on VM vm3 by admin. >2014-08-28 12:33:14,168 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (http--0.0.0.0-8080-2) [101f8991] BaseAsyncTask::startPollingTask: Starting to poll task 5e25b889-c7d0-4546-b615-1ff290d1d78b. >2014-08-28 12:33:14,860 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-59) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-08-28 12:33:14,907 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-59) SPMAsyncTask::PollTask: Polling task 5e25b889-c7d0-4546-b615-1ff290d1d78b (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-28 12:33:14,962 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-59) BaseAsyncTask::onTaskEndSuccess: Task 5e25b889-c7d0-4546-b615-1ff290d1d78b (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-28 12:33:14,964 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-59) CommandAsyncTask::endActionIfNecessary: All tasks of command e9e9397a-b09f-45c9-bbb7-587d53e441fd has ended -> executing endAction >2014-08-28 12:33:14,965 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-59) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: e9e9397a-b09f-45c9-bbb7-587d53e441fd): calling endAction . >2014-08-28 12:33:14,966 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-26) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddDisk, executionIndex: 0 >2014-08-28 12:33:14,987 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (org.ovirt.thread.pool-8-thread-26) [63cd7894] Ending command successfully: org.ovirt.engine.core.bll.AddDiskCommand >2014-08-28 12:33:15,004 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-8-thread-26) [101f8991] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-08-28 12:33:15,011 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-26) [101f8991] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 5431d17c-df3f-420c-b516-031e4ca4cf0a, imageId = 7123c19f-6adc-4690-9bf0-c26078190321), log id: 422ff64c >2014-08-28 12:33:15,036 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-26) [101f8991] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@c35d0065, log id: 422ff64c >2014-08-28 12:33:15,078 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-26) Correlation ID: 63cd7894, Call Stack: null, Custom Event ID: -1, Message: The disk vm3_Disk3 was successfully added to VM vm3. >2014-08-28 12:33:15,079 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-26) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk completed, handling the result. >2014-08-28 12:33:15,080 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-26) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk succeeded, clearing tasks. >2014-08-28 12:33:15,087 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-26) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 5e25b889-c7d0-4546-b615-1ff290d1d78b >2014-08-28 12:33:15,090 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-26) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 5e25b889-c7d0-4546-b615-1ff290d1d78b), log id: 6484d525 >2014-08-28 12:33:15,092 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-26) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=5e25b889-c7d0-4546-b615-1ff290d1d78b), log id: 4cc5a3b1 >2014-08-28 12:33:15,201 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-26) FINISH, HSMClearTaskVDSCommand, log id: 4cc5a3b1 >2014-08-28 12:33:15,202 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-26) FINISH, SPMClearTaskVDSCommand, log id: 6484d525 >2014-08-28 12:33:15,228 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-26) BaseAsyncTask::removeTaskFromDB: Removed task 5e25b889-c7d0-4546-b615-1ff290d1d78b from DataBase >2014-08-28 12:33:15,228 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-26) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity e9e9397a-b09f-45c9-bbb7-587d53e441fd >2014-08-28 12:33:29,788 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (http--0.0.0.0-8080-2) [383585b6] Lock Acquired to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-28 12:33:29,832 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-47) Command 0a2c12f7-8b22-4384-88d3-255369321405 persisting async task placeholder for child command 12fa7b0f-24c9-49aa-bd61-a08ee3708ca3 >2014-08-28 12:33:29,953 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-47) Command 0a2c12f7-8b22-4384-88d3-255369321405 persisting async task placeholder for child command 1d508ec1-48c1-4a1b-8914-a995b53d2566 >2014-08-28 12:33:30,004 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-47) Running command: CreateAllSnapshotsFromVmCommand internal: false. Entities affected : ID: 5b1ee280-4251-489a-899c-8e9dac73c327 Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER >2014-08-28 12:33:30,120 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-47) [672b6298] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-08-28 12:33:30,129 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-47) [672b6298] START, CreateSnapshotVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageSizeInBytes = 10737418240, volumeFormat = COW, newImageId = c8e2a0e9-145e-4ca4-b128-d0f75b5622b7, newImageDescription = , imageId = 26308abd-8540-410e-8018-58cd2157084d, sourceImageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1), log id: 6c0bb274 >2014-08-28 12:33:30,133 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-47) [672b6298] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-28 12:33:30,529 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-47) [672b6298] FINISH, CreateSnapshotVDSCommand, return: c8e2a0e9-145e-4ca4-b128-d0f75b5622b7, log id: 6c0bb274 >2014-08-28 12:33:30,693 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-47) [672b6298] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 0a2c12f7-8b22-4384-88d3-255369321405 >2014-08-28 12:33:30,694 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-47) [672b6298] CommandMultiAsyncTasks::AttachTask: Attaching task 3d39331a-f0a5-42e4-a2c1-71a18709accd to command 0a2c12f7-8b22-4384-88d3-255369321405. >2014-08-28 12:33:30,725 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-47) [672b6298] Adding task 3d39331a-f0a5-42e4-a2c1-71a18709accd (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-28 12:33:30,808 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-47) [39c2e04f] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-08-28 12:33:30,816 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-47) [39c2e04f] START, CreateSnapshotVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 5431d17c-df3f-420c-b516-031e4ca4cf0a, imageSizeInBytes = 1073741824, volumeFormat = COW, newImageId = 9b6f71b4-a5d1-4f94-96a0-f84e9568491a, newImageDescription = , imageId = 7123c19f-6adc-4690-9bf0-c26078190321, sourceImageGroupId = 5431d17c-df3f-420c-b516-031e4ca4cf0a), log id: 62bf1467 >2014-08-28 12:33:30,819 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-47) [39c2e04f] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-08-28 12:33:31,325 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (org.ovirt.thread.pool-8-thread-47) [39c2e04f] FINISH, CreateSnapshotVDSCommand, return: 9b6f71b4-a5d1-4f94-96a0-f84e9568491a, log id: 62bf1467 >2014-08-28 12:33:31,382 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (org.ovirt.thread.pool-8-thread-47) [39c2e04f] CommandMultiAsyncTasks::AttachTask: Attaching task 3a5ccec7-5541-489e-bc37-e96db54367ee to command 0a2c12f7-8b22-4384-88d3-255369321405. >2014-08-28 12:33:31,412 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-47) [39c2e04f] Adding task 3a5ccec7-5541-489e-bc37-e96db54367ee (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-28 12:33:31,544 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-47) Correlation ID: 383585b6, Job ID: 1065275e-c2d3-4144-963f-cd675774771c, Call Stack: null, Custom Event ID: -1, Message: Snapshot 's1' creation for VM 'vm3' was initiated by admin. >2014-08-28 12:33:31,545 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-47) BaseAsyncTask::startPollingTask: Starting to poll task 3d39331a-f0a5-42e4-a2c1-71a18709accd. >2014-08-28 12:33:31,545 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-47) BaseAsyncTask::startPollingTask: Starting to poll task 3a5ccec7-5541-489e-bc37-e96db54367ee. >2014-08-28 12:33:34,966 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-6) Polling and updating Async Tasks: 3 tasks, 2 tasks to poll now >2014-08-28 12:33:34,974 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-6) SPMAsyncTask::PollTask: Polling task 3d39331a-f0a5-42e4-a2c1-71a18709accd (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-28 12:33:34,998 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-6) BaseAsyncTask::onTaskEndSuccess: Task 3d39331a-f0a5-42e4-a2c1-71a18709accd (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-28 12:33:35,000 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-6) Task with DB Task ID 2f23839d-2eb3-417c-a814-c1a8d10bdd74 and VDSM Task ID 3a5ccec7-5541-489e-bc37-e96db54367ee is in state Polling. End action for command 0a2c12f7-8b22-4384-88d3-255369321405 will proceed when all the entitys tasks are completed. >2014-08-28 12:33:35,002 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-6) SPMAsyncTask::PollTask: Polling task 3a5ccec7-5541-489e-bc37-e96db54367ee (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-28 12:33:35,015 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-6) BaseAsyncTask::onTaskEndSuccess: Task 3a5ccec7-5541-489e-bc37-e96db54367ee (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-28 12:33:35,016 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-6) CommandAsyncTask::endActionIfNecessary: All tasks of command 0a2c12f7-8b22-4384-88d3-255369321405 has ended -> executing endAction >2014-08-28 12:33:35,017 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-6) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 0a2c12f7-8b22-4384-88d3-255369321405): calling endAction . >2014-08-28 12:33:35,019 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-45) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction CreateAllSnapshotsFromVm, executionIndex: 0 >2014-08-28 12:33:35,040 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-45) Ending command successfully: org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand >2014-08-28 12:33:35,065 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-45) [39c2e04f] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotCommand >2014-08-28 12:33:35,074 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-45) [39c2e04f] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 5431d17c-df3f-420c-b516-031e4ca4cf0a, imageId = 9b6f71b4-a5d1-4f94-96a0-f84e9568491a), log id: 4bcdbdb4 >2014-08-28 12:33:35,101 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-45) [39c2e04f] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@18ea33cd, log id: 4bcdbdb4 >2014-08-28 12:33:35,139 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (org.ovirt.thread.pool-8-thread-45) [672b6298] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotCommand >2014-08-28 12:33:35,145 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-45) [672b6298] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = b5576e45-7f13-465f-b26f-05d52faf28d1, imageId = c8e2a0e9-145e-4ca4-b128-d0f75b5622b7), log id: 62b62a4d >2014-08-28 12:33:35,171 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-8-thread-45) [672b6298] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@750a5c7f, log id: 62b62a4d >2014-08-28 12:33:35,212 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (org.ovirt.thread.pool-8-thread-45) Lock freed to object EngineLock [exclusiveLocks= key: 5b1ee280-4251-489a-899c-8e9dac73c327 value: VM >, sharedLocks= ] >2014-08-28 12:33:35,238 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-45) Correlation ID: 383585b6, Call Stack: null, Custom Event ID: -1, Message: Snapshot 's1' creation for VM 'vm3' has been completed. >2014-08-28 12:33:35,239 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-45) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type CreateAllSnapshotsFromVm completed, handling the result. >2014-08-28 12:33:35,239 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-45) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type CreateAllSnapshotsFromVm succeeded, clearing tasks. >2014-08-28 12:33:35,247 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-45) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 3d39331a-f0a5-42e4-a2c1-71a18709accd >2014-08-28 12:33:35,247 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-45) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 3d39331a-f0a5-42e4-a2c1-71a18709accd), log id: d8f471c >2014-08-28 12:33:35,249 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-45) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=3d39331a-f0a5-42e4-a2c1-71a18709accd), log id: 39858638 >2014-08-28 12:33:36,120 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-45) FINISH, HSMClearTaskVDSCommand, log id: 39858638 >2014-08-28 12:33:36,121 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-45) FINISH, SPMClearTaskVDSCommand, log id: d8f471c >2014-08-28 12:33:36,183 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-45) BaseAsyncTask::removeTaskFromDB: Removed task 3d39331a-f0a5-42e4-a2c1-71a18709accd from DataBase >2014-08-28 12:33:36,184 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-45) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 3a5ccec7-5541-489e-bc37-e96db54367ee >2014-08-28 12:33:36,185 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-45) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 3a5ccec7-5541-489e-bc37-e96db54367ee), log id: 555e633 >2014-08-28 12:33:36,187 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-45) START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=3a5ccec7-5541-489e-bc37-e96db54367ee), log id: 4a7c2101 >2014-08-28 12:33:36,250 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-45) FINISH, HSMClearTaskVDSCommand, log id: 4a7c2101 >2014-08-28 12:33:36,250 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-45) FINISH, SPMClearTaskVDSCommand, log id: 555e633 >2014-08-28 12:33:36,266 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-45) BaseAsyncTask::removeTaskFromDB: Removed task 3a5ccec7-5541-489e-bc37-e96db54367ee from DataBase >2014-08-28 12:33:36,266 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-45) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 0a2c12f7-8b22-4384-88d3-255369321405 >2014-08-28 12:34:05,061 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-8) Attempting to update VMs/Templates Ovf. >2014-08-28 12:34:05,486 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-8) [7d900ca9] Lock Acquired to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-28 12:34:05,490 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-8) [7d900ca9] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-28 12:34:05,647 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [7d900ca9] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4c16b586-1b20-4f3a-9001-43521e3bb552, imageId = 655fad6a-3526-407e-a7d9-2d5e09102565), log id: 14b117da >2014-08-28 12:34:05,649 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [7d900ca9] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-28 12:34:05,649 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [7d900ca9] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4c16b586-1b20-4f3a-9001-43521e3bb552 > volUUID=655fad6a-3526-407e-a7d9-2d5e09102565 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Wed Aug 13 14:57:40 IDT 2014"} > >2014-08-28 12:34:06,041 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [7d900ca9] FINISH, SetVolumeDescriptionVDSCommand, log id: 14b117da >2014-08-28 12:34:06,290 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] Running command: UploadStreamCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-28 12:34:06,299 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 65a79f96 >2014-08-28 12:34:06,300 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] -- executeVdsBrokerCommand: >2014-08-28 12:34:06,301 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGUID=4c16b586-1b20-4f3a-9001-43521e3bb552 > volUUID=655fad6a-3526-407e-a7d9-2d5e09102565 > size=81,920 > >2014-08-28 12:34:07,929 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] FINISH, UploadStreamVDSCommand, log id: 65a79f96 >2014-08-28 12:34:07,943 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command e3825db2-4009-4dad-aceb-263214a7607a >2014-08-28 12:34:07,944 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] CommandMultiAsyncTasks::AttachTask: Attaching task 2bc5439f-be69-4a14-b3d4-2e1e523ba650 to command e3825db2-4009-4dad-aceb-263214a7607a. >2014-08-28 12:34:07,963 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] Adding task 2bc5439f-be69-4a14-b3d4-2e1e523ba650 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-28 12:34:07,980 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-28 12:34:07,986 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4c16b586-1b20-4f3a-9001-43521e3bb552, imageId = 655fad6a-3526-407e-a7d9-2d5e09102565), log id: c76bb97 >2014-08-28 12:34:07,987 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-28 12:34:07,988 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4c16b586-1b20-4f3a-9001-43521e3bb552 > volUUID=655fad6a-3526-407e-a7d9-2d5e09102565 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Thu Aug 28 12:34:05 IDT 2014","Size":81920} > >2014-08-28 12:34:08,096 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] FINISH, SetVolumeDescriptionVDSCommand, log id: c76bb97 >2014-08-28 12:34:08,107 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4cab3024-3f6d-435a-bf22-a6e7daac3bd9, imageId = 612a0f89-926c-48c5-9804-b258314e4838), log id: 79c048da >2014-08-28 12:34:08,109 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-28 12:34:08,110 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4cab3024-3f6d-435a-bf22-a6e7daac3bd9 > volUUID=612a0f89-926c-48c5-9804-b258314e4838 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Wed Aug 13 14:57:40 IDT 2014"} > >2014-08-28 12:34:08,201 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [3f7df8bf] FINISH, SetVolumeDescriptionVDSCommand, log id: 79c048da >2014-08-28 12:34:08,244 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] Running command: UploadStreamCommand internal: true. Entities affected : ID: d54e6e6c-81f2-41f2-90ab-7e3c2457938a Type: Storage >2014-08-28 12:34:08,246 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 61480bf3 >2014-08-28 12:34:08,247 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] -- executeVdsBrokerCommand: >2014-08-28 12:34:08,247 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGUID=4cab3024-3f6d-435a-bf22-a6e7daac3bd9 > volUUID=612a0f89-926c-48c5-9804-b258314e4838 > size=81,920 > >2014-08-28 12:34:09,917 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] FINISH, UploadStreamVDSCommand, log id: 61480bf3 >2014-08-28 12:34:09,918 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] CommandMultiAsyncTasks::AttachTask: Attaching task c32c2cee-a37f-4fd4-89f2-fba1432e7e22 to command e3825db2-4009-4dad-aceb-263214a7607a. >2014-08-28 12:34:09,942 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] Adding task c32c2cee-a37f-4fd4-89f2-fba1432e7e22 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-28 12:34:09,959 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-28 12:34:09,959 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] Trying to release exclusive lock which does not exist, lock key: d54e6e6c-81f2-41f2-90ab-7e3c2457938aSTORAGE >2014-08-28 12:34:09,960 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] Trying to release exclusive lock which does not exist, lock key: 4c16b586-1b20-4f3a-9001-43521e3bb552DISK >2014-08-28 12:34:09,960 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] Trying to release exclusive lock which does not exist, lock key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9DISK >2014-08-28 12:34:09,960 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-28 12:34:09,962 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = d54e6e6c-81f2-41f2-90ab-7e3c2457938a, imageGroupId = 4cab3024-3f6d-435a-bf22-a6e7daac3bd9, imageId = 612a0f89-926c-48c5-9804-b258314e4838), log id: 604786f1 >2014-08-28 12:34:09,963 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-28 12:34:09,963 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=d54e6e6c-81f2-41f2-90ab-7e3c2457938a > imageGroupGUID=4cab3024-3f6d-435a-bf22-a6e7daac3bd9 > volUUID=612a0f89-926c-48c5-9804-b258314e4838 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"d54e6e6c-81f2-41f2-90ab-7e3c2457938a"}],"Last Updated":"Thu Aug 28 12:34:05 IDT 2014","Size":81920} > >2014-08-28 12:34:10,089 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] FINISH, SetVolumeDescriptionVDSCommand, log id: 604786f1 >2014-08-28 12:34:10,108 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] BaseAsyncTask::startPollingTask: Starting to poll task 2bc5439f-be69-4a14-b3d4-2e1e523ba650. >2014-08-28 12:34:10,108 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-8) [70a2b0bb] BaseAsyncTask::startPollingTask: Starting to poll task c32c2cee-a37f-4fd4-89f2-fba1432e7e22. >2014-08-28 12:34:10,126 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-8) [4e28fb77] Lock Acquired to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-28 12:34:10,130 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-8) [4e28fb77] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-28 12:34:10,152 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [4e28fb77] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 588bbfb6 >2014-08-28 12:34:10,152 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [4e28fb77] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-28 12:34:10,153 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [4e28fb77] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Wed Aug 13 14:57:49 IDT 2014"} > >2014-08-28 12:34:10,265 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [4e28fb77] FINISH, SetVolumeDescriptionVDSCommand, log id: 588bbfb6 >2014-08-28 12:34:10,306 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-28 12:34:10,308 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: fb378a >2014-08-28 12:34:10,309 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] -- executeVdsBrokerCommand: >2014-08-28 12:34:10,309 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > size=102,400 > >2014-08-28 12:34:12,288 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] FINISH, UploadStreamVDSCommand, log id: fb378a >2014-08-28 12:34:12,291 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-8) [2eadb570] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command fded18d7-dea4-4511-8a67-280ade20c935 >2014-08-28 12:34:12,292 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-8) [2eadb570] CommandMultiAsyncTasks::AttachTask: Attaching task 347900e4-b970-468b-80c2-06cd87dca20a to command fded18d7-dea4-4511-8a67-280ade20c935. >2014-08-28 12:34:12,484 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-8) [2eadb570] Adding task 347900e4-b970-468b-80c2-06cd87dca20a (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-28 12:34:12,501 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-28 12:34:12,514 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = af890a8b-f1cc-40c4-959c-3c3bf08008a1, imageId = f9859dec-7bd0-43a0-b1c8-fed950bf3833), log id: 113ff111 >2014-08-28 12:34:12,515 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-28 12:34:12,516 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=af890a8b-f1cc-40c4-959c-3c3bf08008a1 > volUUID=f9859dec-7bd0-43a0-b1c8-fed950bf3833 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Thu Aug 28 12:34:10 IDT 2014","Size":102400} > >2014-08-28 12:34:12,644 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] FINISH, SetVolumeDescriptionVDSCommand, log id: 113ff111 >2014-08-28 12:34:12,677 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 26b8b814 >2014-08-28 12:34:12,679 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-28 12:34:12,680 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Wed Aug 13 14:57:49 IDT 2014"} > >2014-08-28 12:34:12,889 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [2eadb570] FINISH, SetVolumeDescriptionVDSCommand, log id: 26b8b814 >2014-08-28 12:34:12,939 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] Running command: UploadStreamCommand internal: true. Entities affected : ID: 7158eae9-4886-4af7-af0b-d668ecee9e4f Type: Storage >2014-08-28 12:34:12,941 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] START, UploadStreamVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77), log id: 4aeaa39f >2014-08-28 12:34:12,942 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] -- executeVdsBrokerCommand: >2014-08-28 12:34:12,942 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] -- parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > size=102,400 > >2014-08-28 12:34:14,615 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] FINISH, UploadStreamVDSCommand, log id: 4aeaa39f >2014-08-28 12:34:14,618 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-8) [342cbfee] CommandMultiAsyncTasks::AttachTask: Attaching task c728344e-b44b-4d56-954e-2d7449383232 to command fded18d7-dea4-4511-8a67-280ade20c935. >2014-08-28 12:34:14,687 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Adding task c728344e-b44b-4d56-954e-2d7449383232 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-08-28 12:34:14,704 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-28 12:34:14,705 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-28 12:34:14,706 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-28 12:34:14,707 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-28 12:34:14,707 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-28 12:34:14,710 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, storageDomainId = 7158eae9-4886-4af7-af0b-d668ecee9e4f, imageGroupId = 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e, imageId = 124c7f5e-da9c-4ee5-992a-c9e5afb03460), log id: 51033d7c >2014-08-28 12:34:14,711 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-08-28 12:34:14,712 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-00000000037f > sdUUID=7158eae9-4886-4af7-af0b-d668ecee9e4f > imageGroupGUID=46e7e0b6-db7c-4b0b-892c-d6530a6ad36e > volUUID=124c7f5e-da9c-4ee5-992a-c9e5afb03460 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"7158eae9-4886-4af7-af0b-d668ecee9e4f"}],"Last Updated":"Thu Aug 28 12:34:10 IDT 2014","Size":102400} > >2014-08-28 12:34:14,844 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] FINISH, SetVolumeDescriptionVDSCommand, log id: 51033d7c >2014-08-28 12:34:14,853 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-8) [342cbfee] BaseAsyncTask::startPollingTask: Starting to poll task 347900e4-b970-468b-80c2-06cd87dca20a. >2014-08-28 12:34:14,853 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-8) [342cbfee] BaseAsyncTask::startPollingTask: Starting to poll task c728344e-b44b-4d56-954e-2d7449383232. >2014-08-28 12:34:15,020 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-34) Polling and updating Async Tasks: 7 tasks, 4 tasks to poll now >2014-08-28 12:34:15,033 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) SPMAsyncTask::PollTask: Polling task c32c2cee-a37f-4fd4-89f2-fba1432e7e22 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-28 12:34:15,034 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) BaseAsyncTask::onTaskEndSuccess: Task c32c2cee-a37f-4fd4-89f2-fba1432e7e22 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-28 12:34:15,035 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-34) Task with DB Task ID fcaf983e-6eaf-4682-a025-4cffe7493c53 and VDSM Task ID 2bc5439f-be69-4a14-b3d4-2e1e523ba650 is in state Polling. End action for command e3825db2-4009-4dad-aceb-263214a7607a will proceed when all the entitys tasks are completed. >2014-08-28 12:34:15,037 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) SPMAsyncTask::PollTask: Polling task c728344e-b44b-4d56-954e-2d7449383232 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-28 12:34:15,038 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) BaseAsyncTask::onTaskEndSuccess: Task c728344e-b44b-4d56-954e-2d7449383232 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-28 12:34:15,039 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-34) Task with DB Task ID 29d83fb6-a542-4773-a468-a1aed49288ad and VDSM Task ID 347900e4-b970-468b-80c2-06cd87dca20a is in state Polling. End action for command fded18d7-dea4-4511-8a67-280ade20c935 will proceed when all the entitys tasks are completed. >2014-08-28 12:34:15,041 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) SPMAsyncTask::PollTask: Polling task 347900e4-b970-468b-80c2-06cd87dca20a (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-28 12:34:15,042 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) BaseAsyncTask::onTaskEndSuccess: Task 347900e4-b970-468b-80c2-06cd87dca20a (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-28 12:34:15,043 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-34) CommandAsyncTask::endActionIfNecessary: All tasks of command fded18d7-dea4-4511-8a67-280ade20c935 has ended -> executing endAction >2014-08-28 12:34:15,044 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-34) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: fded18d7-dea4-4511-8a67-280ade20c935): calling endAction . >2014-08-28 12:34:15,045 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) SPMAsyncTask::PollTask: Polling task 2bc5439f-be69-4a14-b3d4-2e1e523ba650 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-08-28 12:34:15,045 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-1) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-08-28 12:34:15,046 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-34) BaseAsyncTask::onTaskEndSuccess: Task 2bc5439f-be69-4a14-b3d4-2e1e523ba650 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-08-28 12:34:15,049 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-34) CommandAsyncTask::endActionIfNecessary: All tasks of command e3825db2-4009-4dad-aceb-263214a7607a has ended -> executing endAction >2014-08-28 12:34:15,050 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-34) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: e3825db2-4009-4dad-aceb-263214a7607a): calling endAction . >2014-08-28 12:34:15,052 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-08-28 12:34:15,083 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-08-28 12:34:15,083 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-08-28 12:34:15,084 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-28 12:34:15,085 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] Trying to release exclusive lock which does not exist, lock key: d54e6e6c-81f2-41f2-90ab-7e3c2457938aSTORAGE >2014-08-28 12:34:15,086 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] Trying to release exclusive lock which does not exist, lock key: 4c16b586-1b20-4f3a-9001-43521e3bb552DISK >2014-08-28 12:34:15,087 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] Trying to release exclusive lock which does not exist, lock key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9DISK >2014-08-28 12:34:15,087 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] Lock freed to object EngineLock [exclusiveLocks= key: d54e6e6c-81f2-41f2-90ab-7e3c2457938a value: STORAGE >key: 4c16b586-1b20-4f3a-9001-43521e3bb552 value: DISK >key: 4cab3024-3f6d-435a-bf22-a6e7daac3bd9 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-28 12:34:15,087 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] Trying to release a shared lock for key: 00000002-0002-0002-0002-00000000037fOVF_UPDATE , but lock does not exist >2014-08-28 12:34:15,088 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-08-28 12:34:15,089 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] Trying to release exclusive lock which does not exist, lock key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36eDISK >2014-08-28 12:34:15,090 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-08-28 12:34:15,090 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] Trying to release exclusive lock which does not exist, lock key: af890a8b-f1cc-40c4-959c-3c3bf08008a1DISK >2014-08-28 12:34:15,091 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 2bc5439f-be69-4a14-b3d4-2e1e523ba650 >2014-08-28 12:34:15,092 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] Trying to release exclusive lock which does not exist, lock key: 7158eae9-4886-4af7-af0b-d668ecee9e4fSTORAGE >2014-08-28 12:34:15,092 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 2bc5439f-be69-4a14-b3d4-2e1e523ba650), log id: 1c1839f4 >2014-08-28 12:34:15,093 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] Lock freed to object EngineLock [exclusiveLocks= key: 46e7e0b6-db7c-4b0b-892c-d6530a6ad36e value: DISK >key: af890a8b-f1cc-40c4-959c-3c3bf08008a1 value: DISK >key: 7158eae9-4886-4af7-af0b-d668ecee9e4f value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-00000000037f value: OVF_UPDATE >] >2014-08-28 12:34:15,095 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=2bc5439f-be69-4a14-b3d4-2e1e523ba650), log id: 2599e363 >2014-08-28 12:34:15,095 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-08-28 12:34:15,097 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-08-28 12:34:15,098 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 347900e4-b970-468b-80c2-06cd87dca20a >2014-08-28 12:34:15,099 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = 347900e4-b970-468b-80c2-06cd87dca20a), log id: 3e4f571a >2014-08-28 12:34:15,146 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] FINISH, HSMClearTaskVDSCommand, log id: 2599e363 >2014-08-28 12:34:15,147 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] FINISH, SPMClearTaskVDSCommand, log id: 1c1839f4 >2014-08-28 12:34:15,149 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=347900e4-b970-468b-80c2-06cd87dca20a), log id: 4adf8beb >2014-08-28 12:34:15,200 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] BaseAsyncTask::removeTaskFromDB: Removed task 2bc5439f-be69-4a14-b3d4-2e1e523ba650 from DataBase >2014-08-28 12:34:15,201 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] SPMAsyncTask::ClearAsyncTask: Attempting to clear task c32c2cee-a37f-4fd4-89f2-fba1432e7e22 >2014-08-28 12:34:15,201 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = c32c2cee-a37f-4fd4-89f2-fba1432e7e22), log id: aad9ba8 >2014-08-28 12:34:15,204 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] FINISH, HSMClearTaskVDSCommand, log id: 4adf8beb >2014-08-28 12:34:15,204 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] FINISH, SPMClearTaskVDSCommand, log id: 3e4f571a >2014-08-28 12:34:15,205 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=c32c2cee-a37f-4fd4-89f2-fba1432e7e22), log id: bc63812 >2014-08-28 12:34:15,217 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] BaseAsyncTask::removeTaskFromDB: Removed task 347900e4-b970-468b-80c2-06cd87dca20a from DataBase >2014-08-28 12:34:15,217 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] SPMAsyncTask::ClearAsyncTask: Attempting to clear task c728344e-b44b-4d56-954e-2d7449383232 >2014-08-28 12:34:15,218 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000037f, ignoreFailoverLimit = false, taskId = c728344e-b44b-4d56-954e-2d7449383232), log id: 3caaa400 >2014-08-28 12:34:15,256 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] FINISH, HSMClearTaskVDSCommand, log id: bc63812 >2014-08-28 12:34:15,257 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] FINISH, SPMClearTaskVDSCommand, log id: aad9ba8 >2014-08-28 12:34:15,257 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] START, HSMClearTaskVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, taskId=c728344e-b44b-4d56-954e-2d7449383232), log id: 66f51b1d >2014-08-28 12:34:15,267 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] BaseAsyncTask::removeTaskFromDB: Removed task c32c2cee-a37f-4fd4-89f2-fba1432e7e22 from DataBase >2014-08-28 12:34:15,267 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-50) [7d900ca9] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity e3825db2-4009-4dad-aceb-263214a7607a >2014-08-28 12:34:15,312 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] FINISH, HSMClearTaskVDSCommand, log id: 66f51b1d >2014-08-28 12:34:15,313 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] FINISH, SPMClearTaskVDSCommand, log id: 3caaa400 >2014-08-28 12:34:15,324 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] BaseAsyncTask::removeTaskFromDB: Removed task c728344e-b44b-4d56-954e-2d7449383232 from DataBase >2014-08-28 12:34:15,325 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-1) [4e28fb77] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity fded18d7-dea4-4511-8a67-280ade20c935 >2014-08-28 12:34:34,793 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-87) Setting new tasks map. The map contains now 6 tasks >2014-08-28 12:35:04,822 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-67) Setting new tasks map. The map contains now 4 tasks >2014-08-28 12:35:34,823 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-63) Setting new tasks map. The map contains now 0 tasks >2014-08-28 12:35:34,824 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-63) Cleared all tasks of pool 00000002-0002-0002-0002-00000000037f. >2014-08-28 13:34:14,855 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VMs/Templates Ovf. >2014-08-28 14:34:14,885 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-48) Attempting to update VMs/Templates Ovf. >2014-08-28 15:34:14,924 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-76) Attempting to update VMs/Templates Ovf. >2014-08-28 16:34:14,948 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Attempting to update VMs/Templates Ovf. >2014-08-28 17:34:14,974 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-100) Attempting to update VMs/Templates Ovf. >2014-08-28 18:34:14,998 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-46) Attempting to update VMs/Templates Ovf. >2014-08-28 18:39:38,806 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-76) Command SpmStatusVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, storagePoolId = 00000002-0002-0002-0002-00000000037f) execution failed. Exception: VDSNetworkException: java.net.UnknownHostException: battlestar.tlv.redhat.com >2014-08-28 18:39:38,811 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-76) [7a22acac] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-28 18:39:38,814 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-76) [7a22acac] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain 7158eae9-4886-4af7-af0b-d668ecee9e4f status from Active to Unknown, reason : null >2014-08-28 18:39:38,816 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-76) [7a22acac] Storage Pool 00000002-0002-0002-0002-00000000037f - Updating Storage Domain d54e6e6c-81f2-41f2-90ab-7e3c2457938a status from Active to Unknown, reason : null >2014-08-28 18:39:39,268 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-76) [7a22acac] Correlation ID: 7a22acac, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host h1, Error: Network error during communication with the Host.). >2014-08-31 07:09:20,763 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-77) Start deleteAgedOutAuditLogs >2014-08-31 07:09:20,763 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update VMs/Templates Ovf. >2014-08-31 07:09:20,763 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-55) Start deleteAgedOutCommandEntities >2014-08-31 07:09:21,231 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-55) Finished deleteAgedOutCommandEntities >2014-08-31 07:09:21,331 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-77) Finished deleteAgedOutAuditLogs >2014-08-31 07:09:21,596 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-2) [7d0e0c37] Running command: LogoutUserCommand internal: false. >2014-08-31 07:09:21,597 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (http--0.0.0.0-8080-1) [688d356a] Running command: LogoutUserCommand internal: false. >2014-08-31 07:09:21,654 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-1) [688d356a] Correlation ID: 688d356a, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-31 07:09:21,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (http--0.0.0.0-8080-2) [7d0e0c37] Correlation ID: 7d0e0c37, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-08-31 07:11:45,369 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-79) Command ListVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException >2014-08-31 07:11:45,370 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Host h1 is not responding. It will stay in Connecting state for a grace period of 80 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:11:45,394 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-79) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 80 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:11:45,394 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:416) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.fetchRunningVms(VdsUpdateRunTimeInfo.java:974) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVmStats(VdsUpdateRunTimeInfo.java:923) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:641) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.util.concurrent.TimeoutException > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.get(FutureTask.java:119) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy.invoke(XmlRpcUtils.java:200) [vdsbroker.jar:] > at com.sun.proxy.$Proxy83.list(Unknown Source) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper.list(VdsServerWrapper.java:127) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand.executeVdsBrokerCommand(ListVDSCommand.java:23) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > ... 14 more > >2014-08-31 07:11:45,422 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.util.concurrent.TimeoutException >2014-08-31 07:11:50,501 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:11:50,503 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host h1 is not responding. It will stay in Connecting state for a grace period of 80 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:11:50,509 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:11:50,535 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Failed to refresh VDS , vds = 9c67d903-8424-4cda-8bb1-3d8652b92d77 : h1, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-08-31 07:11:54,263 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-31 07:11:54,265 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Host h1 is not responding. It will stay in Connecting state for a grace period of 80 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:11:54,354 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Server failed to respond, vds_id = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds_name = h1, vm_count = 0, spm_status = SPM, non-responsive_timeout (seconds) = 80, error = java.net.NoRouteToHostException: No route to host >2014-08-31 07:11:54,372 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) >, Custom Event ID: -1, Message: Host h1 is non responsive. >2014-08-31 07:11:54,386 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-8-thread-18) ResourceManager::vdsNotResponding entered for Host 9c67d903-8424-4cda-8bb1-3d8652b92d77, battlestar.tlv.redhat.com >2014-08-31 07:11:54,413 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:11:54,463 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-18) [433138ba] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: 9c67d903-8424-4cda-8bb1-3d8652b92d77 Type: VDS >2014-08-31 07:11:54,480 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-18) [433138ba] Opening SSH Soft Fencing session on host battlestar.tlv.redhat.com >2014-08-31 07:11:57,269 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-8-thread-18) [433138ba] SSH Soft Fencing command failed on host battlestar.tlv.redhat.com: Failed to get the session. >Stdout: >Stderr: >Stacktrace: org.apache.sshd.common.RuntimeSshException: Failed to get the session.: org.apache.sshd.common.RuntimeSshException: Failed to get the session. > at org.apache.sshd.client.future.DefaultConnectFuture.getSession(DefaultConnectFuture.java:43) > at org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:294) [uutils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:87) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:53) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1169) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1300) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1922) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1321) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:346) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:114) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:125) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:135) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:82) [jboss-as-weld-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view8.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$4.run(VdsEventListener.java:243) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishConnect(UnixAsynchronousSocketChannelImpl.java:252) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:198) [rt.jar:1.7.0_25] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:213) [rt.jar:1.7.0_25] > at sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:293) [rt.jar:1.7.0_25] > ... 1 more > >2014-08-31 07:11:57,487 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-18) [7d2a0834] Lock Acquired to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-31 07:11:57,487 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-18) [7d2a0834] Fence failed on vds h1 which is spm of pool 00000002-0002-0002-0002-00000000037f - moving pool to non operational >2014-08-31 07:11:57,494 WARN [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (org.ovirt.thread.pool-8-thread-18) [100669d4] The message key SetStoragePoolStatus is missing from bundles/ExecutionMessages >2014-08-31 07:11:57,528 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-8-thread-18) [100669d4] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000037f Type: StoragePool >2014-08-31 07:11:57,561 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-18) [100669d4] Correlation ID: 100669d4, Job ID: 9b2cb9b1-efd4-4ca3-8d2d-0842c0be2257, Call Stack: null, Custom Event ID: -1, Message: Fencing failed on Storage Pool Manager h1 for Data Center Default. Setting status to Non-Operational. >2014-08-31 07:11:57,569 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-8-thread-18) [100669d4] Lock freed to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-31 07:11:57,570 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-18) [100669d4] Failed to run Fence script on vds:h1. >2014-08-31 07:11:57,589 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-18) [100669d4] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-08-31 07:11:57,590 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-18) [100669d4] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-08-31 07:11:57,591 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-8-thread-18) [100669d4] Trying to release exclusive lock which does not exist, lock key: 9c67d903-8424-4cda-8bb1-3d8652b92d77VDS_FENCE >2014-08-31 07:11:57,591 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-8-thread-18) [100669d4] Lock freed to object EngineLock [exclusiveLocks= key: 9c67d903-8424-4cda-8bb1-3d8652b92d77 value: VDS_FENCE >, sharedLocks= ] >2014-08-31 07:11:59,476 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:11:59,478 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-29) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:11:59,549 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:12:03,281 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-31 07:12:03,283 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:12:03,409 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:12:08,488 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:12:08,489 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:12:08,578 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:12:12,299 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-31 07:12:12,301 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:12:12,364 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:12:17,280 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) [7a22acac] IrsBroker::Failed::GetStoragePoolInfoVDS due to: TimeoutException: >2014-08-31 07:12:17,414 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:12:17,415 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:12:17,483 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:12:22,563 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:12:22,565 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:12:22,668 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:12:27,703 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:12:27,705 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:12:27,802 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:12:32,838 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:12:32,840 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:12:32,881 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:12:37,912 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:12:37,914 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-33) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:12:38,033 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:12:43,069 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:12:43,071 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:12:43,169 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:12:48,204 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:12:48,206 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:12:48,272 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:12:53,308 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:12:53,310 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:12:53,374 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:12:58,407 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:12:58,409 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:12:58,518 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:13:03,555 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:13:03,557 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:13:03,670 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:13:08,708 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:13:08,709 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:13:08,797 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:13:13,831 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:13:13,833 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:13:13,866 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:13:18,902 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:13:18,904 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:13:18,935 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:13:24,015 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:13:24,017 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:13:24,128 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:13:29,203 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:13:29,205 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:13:29,272 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:13:34,310 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:13:34,312 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:13:34,417 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:13:39,451 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:13:39,453 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:13:39,569 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:13:44,602 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:13:44,604 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:13:44,788 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:13:49,823 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:13:49,825 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:13:49,881 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:13:54,918 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:13:54,920 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:13:54,949 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:13:59,981 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:13:59,983 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:00,060 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:14:05,081 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:14:05,083 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:05,213 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:14:10,248 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:14:10,250 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:10,399 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:14:15,431 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:14:15,433 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:15,510 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:14:20,529 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:14:20,531 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:20,612 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:14:25,648 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:14:25,649 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:25,774 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:14:30,810 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:14:30,811 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:30,868 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:14:35,901 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:14:35,903 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:35,996 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:14:41,032 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:14:41,035 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:41,140 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:14:46,180 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:14:46,182 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:46,260 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:46,301 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:14:51,333 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:14:51,335 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:51,412 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:14:55,957 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-31 07:14:55,959 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:14:56,026 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:15:01,058 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:15:01,060 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:15:01,095 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:15:06,127 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:15:06,129 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:15:06,206 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:15:11,238 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:15:11,240 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-68) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:15:11,367 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-68) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:15:16,400 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:15:16,403 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:15:16,519 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:15:21,557 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:15:21,559 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:15:21,655 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:15:26,693 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:15:26,695 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:15:26,816 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:15:31,852 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:15:31,854 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:15:31,885 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:15:36,919 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:15:36,921 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-37) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:15:37,054 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:15:42,088 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:15:42,090 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:15:42,215 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:15:47,249 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:15:47,251 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:15:47,359 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:15:52,393 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:15:52,395 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:15:52,479 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:15:57,521 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:15:57,523 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:15:57,631 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:16:02,664 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:16:02,665 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:16:02,749 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:16:07,775 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:16:07,777 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:16:07,877 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:16:12,914 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:16:12,916 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:16:12,980 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:16:18,012 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:16:18,014 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-37) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:16:18,124 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:16:23,161 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:16:23,163 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:16:23,260 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:16:28,299 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:16:28,301 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:16:28,412 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:16:33,447 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:16:33,449 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:16:33,555 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:16:38,594 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:16:38,595 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:16:38,733 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:16:43,765 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:16:43,766 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:16:43,894 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:16:48,927 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:16:48,928 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:16:49,013 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:16:54,049 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:16:54,051 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:16:54,174 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:16:59,211 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:16:59,213 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:16:59,351 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:17:04,386 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:17:04,388 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:17:04,512 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:17:09,548 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:17:09,550 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:17:09,664 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:17:14,700 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:17:14,702 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:17:14,833 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:17:19,868 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) [7a22acac] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:17:19,870 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) [7a22acac] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:17:19,961 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) [7a22acac] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:17:24,992 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:17:24,993 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:17:25,096 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:17:30,133 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:17:30,135 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:17:30,191 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:17:35,225 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:17:35,226 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:17:35,343 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:17:40,378 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:17:40,380 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-33) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:17:40,504 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:17:45,541 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:17:45,543 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:17:45,573 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:17:50,606 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:17:50,607 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:17:50,644 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-65) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:17:50,661 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:17:55,691 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:17:55,692 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:17:55,738 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:18:00,084 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-31 07:18:00,084 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:18:00,152 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:18:05,172 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:18:05,174 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:18:05,263 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:18:10,300 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:18:10,302 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:18:10,441 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:18:15,476 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:18:15,478 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-29) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:18:15,535 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:18:20,572 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:18:20,574 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-33) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:18:20,679 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:18:25,715 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:18:25,716 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:18:25,823 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:18:30,857 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:18:30,858 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:18:30,892 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:18:35,925 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) [7a22acac] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:18:35,926 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) [7a22acac] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:18:36,036 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) [7a22acac] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:18:41,073 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:18:41,075 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:18:41,181 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) [342cbfee] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:18:46,286 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:18:46,287 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:18:46,383 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:18:51,473 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:18:51,475 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:18:51,576 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:18:56,653 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:18:56,655 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:18:56,745 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:19:01,826 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:19:01,834 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:19:01,939 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:19:07,015 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:19:07,017 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:19:07,149 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:19:12,220 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:19:12,222 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:19:12,376 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:19:17,413 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:19:17,415 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:19:17,495 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:19:22,531 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:19:22,532 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:19:22,597 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:19:27,630 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:19:27,632 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:19:27,741 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:19:32,776 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:19:32,777 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:19:32,860 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:19:37,894 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:19:37,895 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:19:37,971 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:19:43,008 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:19:43,009 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:19:43,157 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:19:48,187 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:19:48,188 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:19:48,277 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:19:53,309 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:19:53,311 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:19:53,454 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:19:58,490 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:19:58,492 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:19:58,614 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:20:03,650 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:20:03,652 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:20:03,792 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:20:08,828 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:20:08,830 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:20:08,870 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:20:13,902 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:20:13,904 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:20:13,973 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:20:19,010 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:20:19,012 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:20:19,067 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:20:24,095 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:20:24,097 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:20:24,203 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:20:29,239 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:20:29,241 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:20:29,296 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:20:34,333 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:20:34,335 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:20:34,473 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:20:39,513 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:20:39,515 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:20:39,617 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:20:44,655 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:20:44,656 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:20:44,787 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:20:49,818 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:20:49,820 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:20:49,872 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:20:54,905 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:20:54,906 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:20:54,974 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-71) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:20:55,007 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:21:00,040 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:21:00,040 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:21:00,094 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:21:04,213 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-31 07:21:04,215 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:21:04,277 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:21:09,311 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:21:09,313 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:21:09,346 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:21:14,380 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:21:14,382 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:21:14,499 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:21:19,530 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:21:19,532 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:21:19,650 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:21:24,687 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:21:24,688 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:21:24,803 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:21:29,841 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:21:29,843 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:21:29,939 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:21:34,973 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:21:35,011 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:21:35,100 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:21:40,136 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:21:40,137 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:21:40,269 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:21:45,303 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:21:45,305 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:21:45,363 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:21:50,403 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:21:50,404 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:21:50,482 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:21:55,519 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:21:55,520 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:21:55,602 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:22:00,637 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:22:00,639 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:22:00,753 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:22:05,788 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:22:05,790 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:22:05,898 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:22:10,930 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:22:10,931 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:22:11,075 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:22:16,097 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:22:16,098 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:22:16,186 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:22:21,222 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:22:21,223 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:22:21,304 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:22:26,323 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:22:26,325 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:22:26,390 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:22:31,422 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:22:31,423 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:22:31,500 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:22:36,519 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:22:36,520 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:22:36,610 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:22:41,627 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:22:41,628 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:22:41,762 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:22:46,778 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:22:46,778 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:22:46,806 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:22:51,822 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:22:51,823 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:22:51,875 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:22:56,891 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:22:56,891 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:22:56,961 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:23:01,978 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:23:01,978 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:02,097 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:23:07,117 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:23:07,118 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:07,249 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:23:12,267 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:23:12,268 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:12,352 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:23:17,368 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:23:17,369 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:17,455 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:23:22,472 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:23:22,473 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:22,557 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:23:27,575 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:23:27,576 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:27,684 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:23:32,703 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:23:32,704 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:32,778 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:23:37,809 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:23:37,810 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:37,889 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:23:42,036 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-08-31 07:23:42,036 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:42,081 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) [342cbfee] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:23:47,099 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:23:47,101 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:47,166 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:23:52,185 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:23:52,187 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-56) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:52,211 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:23:57,237 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:23:57,257 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:57,488 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-57) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:23:57,571 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:24:02,585 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:24:02,587 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:24:02,657 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:24:07,684 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:24:07,685 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:24:07,826 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:24:12,843 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:24:12,844 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:24:12,921 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:24:17,941 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:24:17,941 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:24:18,031 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:24:23,044 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:24:23,045 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:24:23,151 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:24:28,175 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:24:28,176 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:24:28,270 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:24:33,293 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:24:33,294 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:24:33,389 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:24:38,410 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:24:38,411 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:24:38,491 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:24:43,523 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:24:43,524 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:24:43,618 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:24:48,638 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:24:48,639 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:24:48,696 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:24:53,728 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:24:53,729 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:24:53,842 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:24:58,858 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:24:58,859 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:24:58,927 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:25:03,957 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:25:03,958 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:25:04,073 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:25:09,091 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:25:09,093 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:25:09,232 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:25:14,267 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:25:14,268 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:25:14,378 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:25:19,397 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:25:19,400 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:25:19,495 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:25:24,527 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:25:24,528 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:25:24,655 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:25:29,687 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:25:29,688 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:25:29,835 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:25:34,852 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:25:34,853 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:25:34,919 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:25:39,951 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:25:39,952 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:25:40,055 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:25:45,074 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:25:45,076 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:25:45,175 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:25:50,209 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:25:50,210 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:25:50,302 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:25:55,320 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:25:55,322 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:25:55,438 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:26:00,475 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:26:00,476 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:26:00,590 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:26:05,609 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:26:05,611 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:26:05,727 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:26:10,752 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:26:10,754 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:26:10,837 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:26:15,865 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:26:15,866 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:26:15,889 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:26:20,907 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:26:20,908 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-43) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:26:21,000 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:26:26,035 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:26:26,036 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:26:26,156 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:26:31,175 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:26:31,178 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:26:31,270 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:26:36,299 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:26:36,300 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:26:36,422 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:26:41,441 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:26:41,443 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:26:41,547 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:26:46,562 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:26:46,563 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:26:46,643 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:26:51,661 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:26:51,662 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:26:51,746 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:26:56,762 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:26:56,764 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:26:56,849 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:27:01,881 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:27:01,881 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-43) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:01,901 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-43) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:01,917 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:27:06,934 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:27:06,934 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:06,985 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:27:12,004 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:27:12,005 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:12,062 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:27:17,078 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:27:17,079 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:17,131 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:27:22,147 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:27:22,147 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:22,233 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:27:27,249 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:27:27,250 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:27,366 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:27:32,381 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:27:32,392 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:32,477 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:27:37,493 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:27:37,494 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:37,632 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:27:42,649 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:27:42,650 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:42,676 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-46) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:27:47,696 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:27:47,697 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:47,752 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:27:52,771 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:27:52,772 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:52,953 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:27:57,973 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:27:57,974 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:27:58,006 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:28:03,034 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:28:03,035 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:28:03,125 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:28:08,157 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:28:08,159 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:28:08,236 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:28:13,269 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:28:13,271 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:28:13,612 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:28:18,646 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:28:18,648 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-56) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:28:18,780 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:28:23,815 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:28:23,816 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:28:23,858 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:28:28,894 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:28:28,896 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:28:28,968 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:28:34,006 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:28:34,007 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:28:34,079 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:28:39,110 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:28:39,112 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:28:39,199 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:28:44,234 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:28:44,236 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:28:44,351 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:28:49,381 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:28:49,383 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:28:49,453 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:28:54,477 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:28:54,478 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:28:54,572 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:28:59,610 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:28:59,612 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:28:59,733 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:29:04,763 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:29:04,765 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:29:04,844 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:29:09,875 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:29:09,876 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:29:09,947 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:29:14,981 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:29:14,983 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:29:15,124 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:29:20,157 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:29:20,158 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:29:20,251 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:29:25,274 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:29:25,275 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:29:25,370 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:29:30,409 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:29:30,411 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:29:30,513 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:29:35,544 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:29:35,547 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:29:35,657 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:29:40,690 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:29:40,692 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:29:40,810 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:29:45,848 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:29:45,850 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:29:45,879 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:29:50,910 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:29:50,912 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:29:50,990 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:29:56,023 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:29:56,025 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:29:56,125 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:30:01,156 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:30:01,157 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:01,236 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:30:06,266 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:30:06,268 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:06,323 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-96) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:06,355 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:30:11,385 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:30:11,387 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-56) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:11,508 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:30:16,540 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:30:16,541 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:16,635 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:30:21,667 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:30:21,668 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:21,770 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:30:26,795 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:30:26,796 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:26,873 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:30:31,888 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:30:31,889 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:31,965 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:30:36,983 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:30:36,984 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:37,051 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:30:42,072 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:30:42,072 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:42,162 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:30:47,179 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:30:47,180 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:47,272 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:30:52,289 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:30:52,290 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:52,387 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-46) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:30:57,405 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:30:57,405 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:30:57,487 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:31:02,508 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:31:02,508 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:31:02,598 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:31:07,614 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:31:07,615 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:31:07,700 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:31:12,716 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:31:12,717 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:31:12,818 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:31:17,835 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:31:17,836 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:31:17,937 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:31:22,952 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:31:22,953 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:31:23,030 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:31:28,052 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:31:28,053 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:31:28,141 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:31:33,168 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:31:33,169 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:31:33,235 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:31:38,260 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:31:38,260 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:31:38,354 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:31:43,371 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:31:43,372 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:31:43,498 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:31:48,514 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:31:48,515 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:31:48,584 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:31:53,600 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:31:53,601 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:31:53,719 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:31:58,736 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:31:58,736 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:31:58,888 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:32:03,918 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:32:03,919 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:32:03,950 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:32:08,973 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:32:08,974 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:32:09,086 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:32:14,113 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:32:14,113 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:32:14,246 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:32:19,264 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:32:19,266 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:32:19,340 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:32:24,369 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:32:24,369 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:32:24,485 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:32:29,500 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:32:29,501 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:32:29,629 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:32:34,648 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:32:34,650 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:32:34,747 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:32:39,775 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:32:39,775 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:32:39,841 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:32:44,862 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:32:44,863 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:32:44,894 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:32:49,925 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:32:49,926 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:32:49,971 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:32:54,986 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:32:54,987 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:32:55,106 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:33:00,124 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:33:00,125 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:00,192 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:33:05,210 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:33:05,211 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:05,320 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:33:10,335 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:33:10,335 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:10,424 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:10,473 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:33:15,488 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:33:15,489 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:15,550 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:33:20,566 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:33:20,567 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:20,628 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:33:25,647 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:33:25,648 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:25,704 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:33:30,721 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:33:30,722 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:30,824 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:33:35,840 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:33:35,841 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:35,909 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:33:40,925 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:33:40,926 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:41,054 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:33:46,070 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:33:46,071 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:46,122 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:33:51,140 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:33:51,140 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:51,214 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:33:56,235 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:33:56,236 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-33) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:33:56,306 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:34:01,325 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:34:01,326 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:34:01,407 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:34:06,428 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:34:06,429 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:34:06,492 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:34:11,509 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:34:11,510 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:34:11,535 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:34:16,558 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:34:16,559 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:34:16,587 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:34:21,606 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:34:21,607 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:34:21,689 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:34:26,712 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:34:26,713 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:34:26,783 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:34:31,814 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:34:31,829 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:34:31,918 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:34:36,942 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:34:36,943 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:34:37,045 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:34:42,068 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:34:42,069 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:34:42,139 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:34:47,161 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:34:47,162 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:34:47,249 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:34:52,272 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:34:52,273 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:34:52,368 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:34:57,399 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:34:57,400 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:34:57,512 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:35:02,539 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:35:02,540 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:35:02,648 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:35:07,664 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:35:07,665 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:35:07,741 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:35:12,765 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:35:12,766 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:35:12,869 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:35:17,889 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:35:17,889 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:35:17,912 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:35:22,929 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:35:22,930 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:35:23,007 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:35:28,025 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:35:28,026 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:35:28,142 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:35:33,159 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:35:33,160 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:35:33,220 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:35:38,236 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:35:38,237 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-37) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:35:38,372 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:35:43,391 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:35:43,391 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:35:43,542 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:35:48,560 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:35:48,560 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:35:48,685 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:35:53,703 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:35:53,704 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:35:53,838 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:35:58,858 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:35:58,859 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:35:58,890 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:36:03,909 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) [7a22acac] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:36:03,909 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) [7a22acac] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:36:04,027 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) [7a22acac] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:36:09,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:36:09,046 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:36:09,113 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:36:14,129 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:36:14,130 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:36:14,174 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:36:14,207 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:36:19,224 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:36:19,225 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:36:19,391 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:36:24,408 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:36:24,409 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:36:24,485 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:36:29,503 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:36:29,504 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:36:29,571 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:36:34,591 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:36:34,591 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:36:34,723 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:36:39,744 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:36:39,744 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:36:39,851 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:36:44,876 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:36:44,877 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:36:44,913 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:36:49,943 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:36:49,944 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:36:50,031 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:36:55,049 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:36:55,050 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:36:55,125 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:37:00,142 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:37:00,142 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:37:00,261 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:37:05,278 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:37:05,279 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:37:05,372 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:37:10,403 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:37:10,404 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:37:10,516 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:37:15,539 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:37:15,540 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:37:15,651 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:37:20,675 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:37:20,676 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:37:20,786 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:37:25,813 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:37:25,815 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:37:25,855 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:37:30,888 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:37:30,889 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:37:30,916 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:37:35,941 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:37:35,942 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:37:36,050 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:37:41,072 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:37:41,073 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:37:41,215 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:37:46,231 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:37:46,232 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:37:46,258 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:37:51,277 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:37:51,278 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:37:51,369 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:37:56,396 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:37:56,397 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:37:56,487 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:38:01,516 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:38:01,517 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:38:01,572 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:38:06,589 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:38:06,589 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:38:06,667 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:38:11,689 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:38:11,689 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:38:11,819 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:38:16,847 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:38:16,848 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:38:16,888 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:38:21,909 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:38:21,910 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:38:21,949 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:38:26,975 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:38:26,976 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:38:27,034 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-46) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:38:32,055 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:38:32,056 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:38:32,103 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:38:37,122 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:38:37,123 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:38:37,197 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:38:42,214 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:38:42,215 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:38:42,275 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:38:47,291 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:38:47,293 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:38:47,394 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:38:52,424 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:38:52,424 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:38:52,538 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:38:57,558 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:38:57,559 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:38:57,691 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:39:02,710 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:39:02,711 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:02,825 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:39:07,850 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:39:07,851 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:07,878 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-46) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:39:12,904 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:39:12,906 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:12,972 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:39:18,001 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:39:18,002 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:18,058 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-22) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:18,099 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:39:23,117 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:39:23,118 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:23,242 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:39:28,261 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:39:28,263 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:28,320 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:39:33,345 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:39:33,346 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:33,388 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:39:38,409 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:39:38,410 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:38,499 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:39:43,519 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:39:43,520 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:43,624 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:39:48,639 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:39:48,639 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:48,696 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:39:53,711 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:39:53,713 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:53,799 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:39:58,823 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:39:58,830 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:39:58,935 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:40:03,959 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:40:03,960 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:40:04,087 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:40:09,114 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:40:09,114 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:40:09,190 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:40:14,207 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:40:14,208 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:40:14,276 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:40:19,301 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:40:19,302 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:40:19,358 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:40:24,379 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:40:24,381 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:40:24,497 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:40:29,516 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:40:29,517 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:40:29,641 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:40:34,669 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:40:34,670 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:40:34,768 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:40:39,786 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:40:39,787 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:40:39,903 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:40:44,926 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:40:44,927 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:40:45,047 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:40:50,063 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:40:50,064 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:40:50,158 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:40:55,185 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:40:55,186 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:40:55,294 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:41:00,311 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:41:00,312 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:41:00,430 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:41:05,454 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:41:05,454 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:41:05,607 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:41:10,626 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:41:10,627 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:41:10,759 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:41:15,788 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:41:15,789 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:41:15,870 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:41:20,898 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:41:20,898 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:41:21,006 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:41:26,026 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:41:26,027 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:41:26,140 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:41:31,162 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:41:31,163 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:41:31,276 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:41:36,295 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:41:36,325 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:41:36,428 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:41:41,460 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:41:41,462 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:41:41,572 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:41:46,591 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:41:46,592 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:41:46,675 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:41:51,695 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:41:51,695 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:41:51,760 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:41:56,782 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:41:56,783 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:41:56,879 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:42:01,928 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:42:01,929 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:02,057 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:42:07,073 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:42:07,074 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:07,167 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:42:12,184 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:42:12,185 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:12,294 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:42:17,310 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:42:17,311 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:17,396 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:42:22,413 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:42:22,414 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:22,450 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-86) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:22,473 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:42:27,496 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:42:27,496 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:27,617 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:42:32,634 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:42:32,634 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:32,694 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:42:37,710 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:42:37,712 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:37,814 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:42:42,840 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:42:42,841 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:42,875 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:42:47,902 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:42:47,903 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:47,960 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:42:52,988 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:42:52,989 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:53,020 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:42:58,041 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:42:58,042 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:42:58,073 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:43:03,088 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:43:03,089 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:43:03,200 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:43:08,220 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:43:08,220 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:43:08,319 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:43:13,335 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:43:13,337 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:43:13,439 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:43:18,470 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:43:18,471 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:43:18,566 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:43:23,585 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:43:23,586 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:43:23,651 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:43:28,683 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:43:28,685 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:43:28,770 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:43:33,799 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) [7a22acac] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:43:33,800 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) [7a22acac] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:43:33,881 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) [7a22acac] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:43:38,910 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:43:38,911 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:43:38,967 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:43:43,995 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:43:43,996 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:43:44,094 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:43:49,115 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:43:49,116 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:43:49,146 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:43:54,162 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:43:54,163 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-43) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:43:54,282 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:43:59,303 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:43:59,304 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:43:59,434 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:44:04,471 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:44:04,472 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:44:04,595 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:44:09,625 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:44:09,627 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:44:09,755 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:44:14,785 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:44:14,786 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:44:14,941 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:44:19,970 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:44:19,972 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:44:20,076 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:44:25,101 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:44:25,101 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:44:25,229 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:44:30,244 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:44:30,245 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:44:30,339 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:44:35,355 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:44:35,356 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:44:35,474 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:44:40,510 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:44:40,511 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:44:40,609 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:44:45,640 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:44:45,642 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:44:45,728 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:44:50,754 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:44:50,756 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:44:50,838 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:44:55,854 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:44:55,855 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:44:55,956 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:45:00,972 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:45:00,973 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:01,076 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:45:06,094 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:45:06,095 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:06,186 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:45:11,206 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:45:11,207 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:11,296 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:45:16,332 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:45:16,333 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:16,398 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:45:21,442 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:45:21,443 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:21,542 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:45:26,574 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:45:26,576 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:26,645 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-75) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:26,703 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:45:31,731 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:45:31,733 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:31,822 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:45:36,854 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:45:36,855 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:36,884 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:45:41,916 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:45:41,917 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:42,052 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:45:47,071 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:45:47,071 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:47,163 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:45:52,184 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:45:52,185 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:52,282 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:45:57,309 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:45:57,310 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:45:57,337 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:46:02,369 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:46:02,370 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:46:02,497 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:46:07,526 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:46:07,527 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:46:07,667 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:46:12,698 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:46:12,700 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:46:12,769 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:46:17,803 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:46:17,803 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:46:17,896 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:46:22,915 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:46:22,916 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:46:22,998 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:46:28,025 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:46:28,027 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:46:28,100 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:46:33,118 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:46:33,118 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:46:33,236 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:46:38,256 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:46:38,257 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:46:38,372 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-46) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:46:43,406 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:46:43,407 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:46:43,524 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:46:48,551 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:46:48,552 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:46:48,685 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:46:53,706 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:46:53,708 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-37) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:46:53,837 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:46:58,867 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:46:58,868 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:46:58,897 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:47:03,925 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:47:03,927 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-68) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:47:04,041 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-68) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:47:09,058 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:47:09,059 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:47:09,160 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:47:14,177 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:47:14,177 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:47:14,311 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:47:19,328 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:47:19,329 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:47:19,429 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:47:24,445 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:47:24,445 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:47:24,557 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:47:29,574 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:47:29,575 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:47:29,684 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:47:34,702 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:47:34,703 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:47:34,769 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:47:39,786 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:47:39,786 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:47:39,888 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:47:44,905 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:47:44,906 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:47:45,123 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:47:50,142 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:47:50,143 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:47:50,175 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:47:55,197 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:47:55,197 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:47:55,260 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-46) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:48:00,278 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:48:00,279 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:00,437 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:48:05,456 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:48:05,457 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:05,573 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:48:10,598 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:48:10,600 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:10,717 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:48:15,743 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:48:15,743 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:15,859 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:48:20,878 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:48:20,879 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:20,905 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:48:25,923 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:48:25,924 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:25,991 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:48:31,011 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:48:31,013 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:31,086 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:31,144 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:48:36,171 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:48:36,171 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:36,311 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:48:41,327 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:48:41,328 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:41,442 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) [342cbfee] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:48:46,459 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:48:46,460 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:46,511 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:48:51,529 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:48:51,530 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:51,589 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:48:56,617 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:48:56,617 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:48:56,716 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:49:01,731 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:49:01,732 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:49:01,802 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:49:06,818 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:49:06,819 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:49:06,921 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:49:11,938 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:49:11,938 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:49:12,006 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:49:17,022 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:49:17,023 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:49:17,125 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:49:22,143 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:49:22,144 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:49:22,227 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:49:27,247 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:49:27,248 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:49:27,303 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:49:32,319 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:49:32,319 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:49:32,397 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:49:37,413 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:49:37,413 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:49:37,540 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:49:42,557 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:49:42,557 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:49:42,652 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:49:47,668 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:49:47,669 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:49:47,753 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:49:52,770 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:49:52,771 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:49:52,838 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:49:57,859 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:49:57,860 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:49:57,889 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:50:02,908 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:50:02,909 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-37) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:50:02,984 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:50:08,011 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:50:08,012 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:50:08,137 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:50:13,156 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:50:13,156 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:50:13,262 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:50:18,279 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:50:18,280 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:50:18,391 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:50:23,407 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:50:23,408 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:50:23,501 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:50:28,517 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:50:28,518 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:50:28,602 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:50:33,619 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:50:33,620 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:50:33,705 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:50:38,721 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:50:38,721 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:50:38,823 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:50:43,840 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:50:43,840 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:50:43,867 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:50:48,885 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:50:48,886 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:50:48,953 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:50:53,973 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:50:53,974 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:50:54,098 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:50:59,126 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:50:59,127 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:50:59,216 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:51:04,244 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:51:04,245 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:51:04,357 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:51:09,374 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:51:09,375 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:51:09,444 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:51:14,467 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:51:14,468 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:51:14,537 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:51:19,556 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:51:19,557 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:51:19,640 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:51:24,658 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:51:24,659 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:51:24,725 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:51:29,743 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:51:29,744 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:51:29,851 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:51:34,869 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:51:34,870 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:51:34,895 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-62) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:51:34,912 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:51:39,934 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:51:39,935 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:51:40,105 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:51:45,120 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:51:45,121 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:51:45,191 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:51:50,207 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:51:50,208 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:51:50,285 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:51:55,302 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:51:55,303 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:51:55,378 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:52:00,397 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:52:00,397 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:52:00,596 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:52:05,613 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:52:05,613 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:52:05,772 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:52:10,790 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:52:10,791 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:52:10,882 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:52:15,903 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:52:15,904 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:52:15,986 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:52:21,000 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:52:21,001 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:52:21,062 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:52:26,080 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:52:26,081 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:52:26,156 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:52:31,175 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:52:31,176 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:52:31,374 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:52:36,392 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:52:36,407 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-37) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:52:36,502 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:52:41,519 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:52:41,520 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-56) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:52:41,629 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:52:46,651 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:52:46,652 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:52:46,690 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:52:51,712 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:52:51,712 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:52:51,775 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:52:56,795 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:52:56,796 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:52:56,894 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:53:01,914 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:53:01,915 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:53:02,005 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:53:07,025 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:53:07,025 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:53:07,081 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:53:12,099 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:53:12,099 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:53:12,200 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:53:17,216 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:53:17,217 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:53:17,286 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:53:22,302 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:53:22,303 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:53:22,397 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:53:27,413 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:53:27,414 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:53:27,532 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:53:32,548 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:53:32,582 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:53:32,676 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:53:37,698 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:53:37,699 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:53:37,750 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:53:42,766 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:53:42,767 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:53:42,840 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:53:47,858 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:53:47,858 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:53:47,909 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:53:52,928 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:53:52,929 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:53:52,996 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:53:58,016 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:53:58,017 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:53:58,048 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:54:03,066 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:54:03,066 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:03,092 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:54:08,108 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:54:08,109 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:08,261 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:54:13,277 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:54:13,278 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:13,454 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:54:18,474 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:54:18,474 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:18,556 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:54:23,572 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:54:23,573 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:23,642 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:54:28,662 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:54:28,663 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:28,760 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:54:33,781 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:54:33,782 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:33,836 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:54:38,857 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:54:38,857 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:38,959 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:38,983 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:54:44,001 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:54:44,001 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:44,103 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:54:49,120 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:54:49,120 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:49,188 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:54:54,207 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:54:54,208 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:54,282 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:54:59,305 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:54:59,305 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:54:59,409 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:55:04,426 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:55:04,427 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:55:04,539 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:55:09,560 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:55:09,561 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:55:09,692 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:55:14,711 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:55:14,712 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:55:14,825 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:55:19,843 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:55:19,843 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:55:19,870 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:55:24,885 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) [7a22acac] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:55:24,886 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) [7a22acac] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:55:24,963 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) [7a22acac] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:55:29,981 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:55:29,982 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:55:30,107 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:55:35,127 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:55:35,128 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:55:35,235 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:55:40,256 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:55:40,257 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:55:40,371 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:55:45,389 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:55:45,389 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:55:45,499 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:55:50,518 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:55:50,519 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:55:50,609 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:55:55,625 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:55:55,626 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:55:55,737 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:56:00,758 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:56:00,759 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:56:00,864 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:56:05,881 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:56:05,881 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:56:05,925 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:56:10,941 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:56:10,942 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:56:11,010 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:56:16,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:56:16,029 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:56:16,145 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:56:21,166 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:56:21,167 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:56:21,240 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:56:26,256 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:56:26,257 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-56) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:56:26,366 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:56:31,385 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:56:31,387 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:56:31,492 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:56:36,512 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:56:36,513 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:56:36,613 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:56:41,629 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:56:41,629 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:56:41,699 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:56:46,721 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:56:46,721 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:56:46,802 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:56:51,821 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:56:51,822 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:56:51,845 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:56:56,862 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:56:56,863 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:56:56,956 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:57:01,980 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:57:01,980 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:02,033 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:57:07,049 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:57:07,050 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-56) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:07,118 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:57:12,136 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:57:12,137 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:12,243 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:57:17,260 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:57:17,261 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:17,320 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:57:22,339 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) [7a22acac] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:57:22,339 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) [7a22acac] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:22,421 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) [7a22acac] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:57:27,450 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:57:27,451 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:27,540 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:57:32,559 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:57:32,560 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:32,675 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:57:37,699 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:57:37,700 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:37,761 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:57:42,777 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:57:42,778 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:42,831 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-44) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:42,848 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:57:47,866 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:57:47,867 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:47,934 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:57:52,957 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:57:52,958 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:53,028 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:57:58,059 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:57:58,059 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:57:58,138 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:58:03,168 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:58:03,169 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:58:03,249 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:58:08,274 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:58:08,275 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:58:08,359 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:58:13,376 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:58:13,377 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:58:13,469 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:58:18,487 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:58:18,488 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:58:18,589 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:58:23,604 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:58:23,605 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-56) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:58:23,683 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:58:28,703 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:58:28,704 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:58:28,852 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:58:33,873 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:58:33,874 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:58:33,896 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:58:38,912 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:58:38,913 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:58:39,024 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:58:44,053 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:58:44,054 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:58:44,182 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:58:49,203 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:58:49,204 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:58:49,262 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:58:54,285 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:58:54,286 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:58:54,414 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:58:59,432 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:58:59,433 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:58:59,533 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:59:04,550 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:59:04,551 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:59:04,652 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:59:09,669 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:59:09,669 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:59:09,779 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:59:14,797 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:59:14,798 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:59:14,890 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:59:19,919 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:59:19,920 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:59:20,025 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:59:25,052 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:59:25,053 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:59:25,152 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:59:30,171 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:59:30,172 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:59:30,228 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:59:35,247 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:59:35,248 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:59:35,306 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:59:40,325 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:59:40,326 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:59:40,466 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:59:45,483 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:59:45,484 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:59:45,602 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:59:50,618 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:59:50,619 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:59:50,646 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 07:59:55,663 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 07:59:55,664 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 07:59:55,782 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:00:00,798 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:00:00,821 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:00,926 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:00:05,954 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:00:05,954 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:06,086 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:00:11,107 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:00:11,107 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:11,214 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:00:16,241 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:00:16,242 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:16,309 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:00:21,326 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:00:21,327 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:21,403 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:00:26,419 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:00:26,420 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:26,496 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:00:31,510 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) [7a22acac] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:00:31,510 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) [7a22acac] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:31,581 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) [7a22acac] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:00:36,600 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:00:36,601 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:36,734 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:00:41,750 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:00:41,751 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:41,870 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:00:46,893 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:00:46,894 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-33) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:46,914 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:46,931 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:00:51,953 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:00:51,954 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:52,042 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:00:57,063 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:00:57,064 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:00:57,178 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:01:02,195 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:01:02,196 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:01:02,289 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:01:07,311 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:01:07,312 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:01:07,449 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:01:12,472 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:01:12,473 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:01:12,709 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:01:17,724 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:01:17,725 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:01:17,811 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:01:22,829 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:01:22,831 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:01:22,873 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:01:27,907 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:01:27,908 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:01:27,958 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:01:32,989 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:01:33,013 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:01:33,103 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:01:38,132 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:01:38,133 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:01:38,271 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:01:43,298 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:01:43,299 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:01:43,424 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:01:48,446 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:01:48,446 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:01:48,518 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:01:53,532 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:01:53,533 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:01:53,612 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:01:58,628 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:01:58,629 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:01:58,748 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:02:03,765 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:02:03,765 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:02:03,866 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:02:08,889 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:02:08,891 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:02:08,919 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:02:13,941 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:02:13,942 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:02:14,071 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:02:19,090 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:02:19,092 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:02:19,198 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:02:24,230 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:02:24,231 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:02:24,309 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:02:29,332 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:02:29,334 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:02:29,452 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:02:34,476 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:02:34,477 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:02:34,597 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:02:39,613 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:02:39,614 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:02:39,725 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:02:44,741 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:02:44,742 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:02:44,843 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:02:49,868 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:02:49,869 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:02:49,912 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:02:54,931 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:02:54,932 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:02:55,056 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:03:00,073 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:03:00,075 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:00,208 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:03:05,225 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:03:05,226 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:05,329 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:03:10,348 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:03:10,350 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:10,472 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:03:15,490 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:03:15,491 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:15,607 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:03:20,623 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:03:20,624 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:20,717 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:03:25,734 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:03:25,735 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:25,844 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:03:30,860 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:03:30,861 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:30,914 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:03:35,934 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:03:35,936 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:36,042 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:03:41,077 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:03:41,079 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:41,219 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:03:46,245 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:03:46,246 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:46,306 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:03:51,328 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:03:51,328 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:51,409 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:51,441 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:03:56,459 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:03:56,460 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:03:56,581 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:04:01,598 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:04:01,599 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:04:01,713 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:04:06,730 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:04:06,730 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:04:06,845 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:04:11,865 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:04:11,866 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:04:11,893 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:04:16,913 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:04:16,914 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:04:16,971 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:04:21,994 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:04:21,996 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:04:22,065 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:04:27,096 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:04:27,097 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:04:27,167 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:04:32,193 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:04:32,195 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:04:32,318 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:04:37,344 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:04:37,345 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:04:37,460 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:04:42,481 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:04:42,481 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:04:42,616 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:04:47,643 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:04:47,643 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:04:47,718 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:04:52,735 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:04:52,735 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:04:52,850 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:04:57,873 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:04:57,874 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:04:57,907 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:05:02,924 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:05:02,926 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:05:03,042 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:05:08,060 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:05:08,061 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:05:08,128 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:05:13,151 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:05:13,152 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:05:13,230 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:05:18,254 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:05:18,255 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:05:18,348 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:05:23,364 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:05:23,365 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:05:23,476 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:05:28,494 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:05:28,494 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:05:28,718 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:05:33,734 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:05:33,764 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:05:33,878 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:05:38,899 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:05:38,901 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:05:38,932 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:05:43,958 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:05:43,959 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:05:44,101 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:05:49,126 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:05:49,128 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:05:49,221 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:05:54,242 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:05:54,243 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:05:54,324 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:05:59,346 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:05:59,348 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:05:59,452 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:06:04,469 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:06:04,470 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:06:04,580 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:06:09,597 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:06:09,598 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:06:09,741 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:06:14,760 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:06:14,761 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:06:14,901 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:06:19,926 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:06:19,927 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:06:19,954 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:06:24,974 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:06:24,975 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:06:25,098 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:06:30,119 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:06:30,121 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:06:30,250 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:06:35,272 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:06:35,273 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:06:35,411 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:06:40,430 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:06:40,431 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:06:40,558 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:06:45,579 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:06:45,579 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:06:45,699 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:06:50,718 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:06:50,719 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:06:50,802 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:06:55,819 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:06:55,820 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:06:55,863 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-40) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:06:55,904 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:07:00,929 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:07:00,931 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:07:01,040 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:07:06,064 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:07:06,065 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:07:06,193 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:07:11,211 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:07:11,213 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:07:11,320 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:07:16,342 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:07:16,343 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:07:16,473 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:07:21,489 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:07:21,490 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:07:21,567 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:07:26,584 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:07:26,585 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:07:26,686 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:07:31,705 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:07:31,706 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-29) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:07:31,797 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:07:36,813 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:07:36,813 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:07:36,891 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:07:41,910 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:07:41,911 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:07:42,011 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:07:47,038 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:07:47,040 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:07:47,179 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:07:52,201 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:07:52,202 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:07:52,289 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:07:57,312 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:07:57,313 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:07:57,458 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:08:02,480 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:08:02,481 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:08:02,610 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:08:07,626 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:08:07,627 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:08:07,759 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:08:12,776 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:08:12,777 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:08:12,898 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:08:17,921 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:08:17,923 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:08:18,009 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:08:23,027 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:08:23,029 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:08:23,095 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:08:28,113 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:08:28,115 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:08:28,247 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:08:33,276 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:08:33,277 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:08:33,392 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:08:38,412 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:08:38,413 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:08:38,528 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:08:43,543 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:08:43,544 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:08:43,664 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:08:48,685 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:08:48,686 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:08:48,775 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:08:53,791 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:08:53,792 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:08:53,885 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:08:58,909 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:08:58,910 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:08:58,971 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:09:03,993 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:09:03,994 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:09:04,107 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:09:09,133 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:09:09,135 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:09:09,251 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:09:14,278 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:09:14,280 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:09:14,395 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:09:19,419 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:09:19,420 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:09:19,547 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:09:20,970 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-41) Attempting to update VMs/Templates Ovf. >2014-08-31 08:09:24,563 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:09:24,564 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:09:24,616 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:09:29,632 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:09:29,633 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:09:29,752 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:09:34,767 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:09:34,768 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:09:34,888 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:09:39,916 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:09:39,917 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:09:39,950 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:09:44,978 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:09:44,980 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:09:45,111 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:09:50,143 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:09:50,145 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:09:50,196 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:09:55,217 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) [7a22acac] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:09:55,219 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) [7a22acac] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:09:55,248 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) [7a22acac] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:10:00,275 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:10:00,276 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:00,334 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-36) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:00,375 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:10:05,391 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:10:05,391 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:05,512 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:10:10,528 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:10:10,529 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:10,664 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:10:15,689 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:10:15,690 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:15,833 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:10:20,850 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:10:20,851 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:20,894 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:10:25,917 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:10:25,919 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:26,020 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:10:31,050 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:10:31,052 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:31,139 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) [342cbfee] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:10:36,168 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:10:36,169 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:36,275 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:10:41,298 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:10:41,299 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:41,428 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:10:46,444 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:10:46,445 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:46,506 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:10:51,524 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:10:51,525 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:51,592 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:10:56,606 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:10:56,607 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:10:56,687 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:11:01,710 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:11:01,712 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:11:01,767 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:11:06,785 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:11:06,785 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:11:06,909 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:11:11,925 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:11:11,926 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:11:11,979 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:11:17,010 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:11:17,010 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:11:17,106 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:11:22,134 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:11:22,134 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:11:22,217 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:11:27,240 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:11:27,241 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:11:27,364 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:11:32,381 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:11:32,382 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:11:32,489 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:11:37,504 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:11:37,505 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:11:37,624 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:11:42,640 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:11:42,641 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:11:42,777 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:11:47,801 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:11:47,802 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:11:47,880 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:11:52,913 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:11:52,914 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:11:53,023 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:11:58,042 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:11:58,044 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:11:58,142 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:12:03,173 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:12:03,174 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:12:03,295 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:12:08,318 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:12:08,319 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:12:08,389 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:12:13,406 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:12:13,407 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:12:13,514 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:12:18,529 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:12:18,529 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:12:18,620 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:12:23,639 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:12:23,640 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:12:23,739 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:12:28,754 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:12:28,755 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:12:28,841 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:12:33,858 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:12:33,859 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:12:33,927 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:12:38,952 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:12:38,954 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-33) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:12:39,088 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:12:44,120 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:12:44,121 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:12:44,224 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:12:49,251 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:12:49,252 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:12:49,392 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:12:54,412 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:12:54,413 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:12:54,512 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:12:59,527 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:12:59,528 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:12:59,647 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:13:04,668 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:13:04,668 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:13:04,759 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-26) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:13:04,817 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:13:09,834 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:13:09,835 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:13:09,878 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:13:14,897 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:13:14,898 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:13:15,031 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:13:20,061 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:13:20,062 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:13:20,249 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:13:25,273 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:13:25,274 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:13:25,350 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:13:30,365 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:13:30,365 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:13:30,502 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:13:35,518 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:13:35,518 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:13:35,638 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:13:40,656 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:13:40,657 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:13:40,775 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:13:45,795 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:13:45,812 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:13:45,911 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:13:50,937 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:13:50,939 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:13:50,994 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:13:56,014 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:13:56,015 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-33) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:13:56,082 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:14:01,106 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:14:01,107 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:14:01,217 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:14:06,237 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:14:06,238 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:14:06,345 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:14:11,365 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:14:11,366 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:14:11,498 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:14:16,515 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:14:16,515 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:14:16,602 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:14:21,618 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:14:21,619 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:14:21,729 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:14:26,753 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:14:26,754 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-37) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:14:26,880 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:14:31,898 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:14:31,899 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:14:31,925 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:14:36,946 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:14:36,947 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:14:37,086 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:14:42,118 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:14:42,119 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:14:42,266 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:14:47,285 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:14:47,286 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:14:47,390 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:14:52,407 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:14:52,408 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:14:52,493 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:14:57,512 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:14:57,513 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:14:57,620 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) [342cbfee] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:15:02,637 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:15:02,638 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:15:02,730 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:15:07,746 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:15:07,747 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-43) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:15:07,849 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:15:12,871 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:15:12,872 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:15:12,902 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:15:17,935 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:15:17,937 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:15:18,021 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:15:23,055 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:15:23,057 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:15:23,083 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:15:28,112 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:15:28,113 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:15:28,251 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:15:33,280 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:15:33,281 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:15:33,395 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:15:38,410 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:15:38,410 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:15:38,548 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:15:43,564 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:15:43,565 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:15:43,676 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:15:48,693 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:15:48,693 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:15:48,770 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:15:53,791 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:15:53,792 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:15:53,883 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:15:58,904 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:15:58,906 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:15:59,010 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:16:04,037 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:16:04,065 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:16:04,146 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:16:09,173 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:16:09,175 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:16:09,258 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-83) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:16:09,306 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:16:14,331 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:16:14,333 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:16:14,568 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:16:19,586 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:16:19,587 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:16:19,694 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:16:24,718 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:16:24,718 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:16:24,837 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:16:29,858 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:16:29,860 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:16:29,924 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:16:34,952 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:16:34,954 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:16:35,069 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:16:40,098 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:16:40,099 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:16:40,221 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:16:45,253 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:16:45,255 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:16:45,385 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:16:50,400 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:16:50,401 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:16:50,484 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:16:55,501 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:16:55,502 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:16:55,743 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:17:00,764 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:17:00,765 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:17:00,945 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:17:05,982 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:17:05,983 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:17:06,040 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:17:11,065 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:17:11,066 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:17:11,135 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:17:16,161 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:17:16,162 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:17:16,270 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:17:21,287 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:17:21,288 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:17:21,381 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:17:26,399 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:17:26,400 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:17:26,517 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:17:31,532 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:17:31,533 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:17:31,586 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:17:36,601 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:17:36,602 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:17:36,722 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:17:41,739 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:17:41,739 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:17:41,850 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:17:46,867 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:17:46,868 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:17:46,902 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:17:51,924 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:17:51,925 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:17:51,996 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-46) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:17:57,021 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:17:57,023 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:17:57,091 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:18:02,118 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:18:02,119 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:18:02,243 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:18:07,275 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:18:07,276 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:18:07,395 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:18:12,413 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:18:12,413 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:18:12,532 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:18:17,548 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:18:17,549 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:18:17,634 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:18:22,658 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:18:22,659 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:18:22,752 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:18:27,776 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:18:27,777 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:18:27,863 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:18:32,883 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:18:32,884 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:18:32,915 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:18:37,950 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:18:37,968 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:18:38,044 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:18:43,068 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:18:43,069 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:18:43,174 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:18:48,190 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:18:48,191 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-29) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:18:48,291 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:18:53,312 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:18:53,313 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:18:53,385 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:18:58,401 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:18:58,401 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:18:58,504 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:19:03,522 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:19:03,523 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:03,614 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:19:08,630 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:19:08,631 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:08,705 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:19:13,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:19:13,727 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:13,756 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-88) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:13,814 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:19:18,832 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:19:18,833 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:18,865 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:19:23,881 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:19:23,882 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:23,917 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:19:28,934 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:19:28,934 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:28,984 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:19:34,002 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:19:34,002 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:34,070 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:19:39,086 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:19:39,086 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:39,113 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:19:44,133 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:19:44,134 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:44,265 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:19:49,287 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:19:49,288 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:49,351 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:19:54,365 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:19:54,366 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:54,419 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:19:59,442 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:19:59,442 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:19:59,504 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:20:04,521 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:20:04,522 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:20:04,590 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:20:09,608 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:20:09,609 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:20:09,717 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:20:14,734 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:20:14,735 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:20:14,802 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:20:19,824 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:20:19,860 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:20:19,921 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:20:24,940 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:20:24,941 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:20:25,040 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:20:30,067 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:20:30,069 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:20:30,174 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:20:35,204 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:20:35,205 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:20:35,335 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:20:40,349 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:20:40,350 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-29) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:20:40,471 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:20:45,488 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:20:45,489 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:20:45,573 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:20:50,590 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:20:50,590 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:20:50,692 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:20:55,709 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:20:55,710 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:20:55,786 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:21:00,801 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:21:00,810 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:21:00,897 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:21:05,923 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:21:05,924 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:21:06,042 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:21:11,075 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:21:11,076 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:21:11,236 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:21:16,254 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:21:16,255 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:21:16,338 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:21:21,363 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:21:21,364 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-29) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:21:21,465 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:21:26,482 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:21:26,482 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:21:26,583 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:21:31,600 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:21:31,601 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:21:31,743 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:21:36,762 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:21:36,762 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:21:36,880 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:21:41,911 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:21:41,912 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:21:42,007 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:21:47,036 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:21:47,037 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:21:47,192 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:21:52,210 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:21:52,212 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:21:52,303 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) [342cbfee] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:21:57,319 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:21:57,320 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:21:57,405 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:22:02,421 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:22:02,422 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-29) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:02,532 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:22:07,549 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:22:07,550 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:07,594 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:22:12,611 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:22:12,612 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:12,705 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:22:17,720 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:22:17,721 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:17,785 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-66) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:17,815 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:22:22,838 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:22:22,854 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:22,909 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:22:27,932 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:22:27,934 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:28,029 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:22:33,057 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:22:33,059 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:33,172 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:22:38,200 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:22:38,201 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:38,312 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:22:43,328 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:22:43,329 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:43,369 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:22:48,386 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:22:48,386 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:48,429 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:22:53,451 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:22:53,451 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:53,515 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:22:58,531 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:22:58,532 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:22:58,617 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:23:03,634 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:23:03,634 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:23:03,753 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:23:08,774 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:23:08,775 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:23:08,889 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:23:13,911 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:23:13,914 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:23:13,942 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:23:18,967 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:23:18,968 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:23:19,052 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:23:24,074 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:23:24,075 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:23:24,179 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:23:29,212 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:23:29,213 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:23:29,289 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:23:34,307 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:23:34,308 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:23:34,415 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:23:39,433 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:23:39,434 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:23:39,561 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:23:44,579 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:23:44,580 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:23:44,713 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:23:49,731 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:23:49,732 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:23:49,781 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:23:54,798 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:23:54,798 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:23:54,869 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:23:59,893 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:23:59,895 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-56) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:23:59,980 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:24:04,998 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:24:04,999 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:24:05,049 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:24:10,071 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:24:10,073 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:24:10,153 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:24:15,180 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:24:15,181 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:24:15,297 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) [66ec1c9b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:24:20,316 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:24:20,317 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:24:20,466 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:24:25,482 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:24:25,483 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:24:25,510 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:24:30,528 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:24:30,528 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-43) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:24:30,596 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:24:35,620 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:24:35,621 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:24:35,691 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:24:40,709 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:24:40,710 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:24:40,802 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:24:45,819 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:24:45,820 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:24:45,855 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:24:50,884 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:24:50,886 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:24:50,998 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:24:56,016 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:24:56,016 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:24:56,093 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:25:01,114 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:25:01,115 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:01,228 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:25:06,245 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:25:06,246 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:06,364 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:25:11,394 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:25:11,395 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:11,534 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:25:16,550 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:25:16,550 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:16,628 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:25:21,644 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:25:21,645 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:21,706 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-52) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:21,739 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:25:26,757 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:25:26,757 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:26,867 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:25:31,889 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:25:31,891 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:31,953 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:25:36,970 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:25:36,971 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:37,105 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:25:42,135 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:25:42,136 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:42,266 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:25:47,284 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:25:47,285 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:47,352 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:25:52,369 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:25:52,370 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:52,463 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:25:57,478 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:25:57,479 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:25:57,574 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:26:02,591 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:26:02,592 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:26:02,702 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:26:07,717 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:26:07,717 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:26:07,821 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:26:12,845 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:26:12,846 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:26:12,891 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:26:17,915 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:26:17,916 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:26:17,993 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) [342cbfee] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:26:23,010 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:26:23,011 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:26:23,104 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:26:28,137 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:26:28,138 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:26:28,239 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:26:33,256 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:26:33,257 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:26:33,366 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:26:38,388 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:26:38,389 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:26:38,511 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:26:43,528 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:26:43,529 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:26:43,644 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:26:48,658 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:26:48,659 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:26:48,766 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:26:53,783 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:26:53,784 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:26:53,910 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:26:58,932 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:26:58,933 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-37) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:26:59,004 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:27:04,030 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:27:04,031 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:27:04,114 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:27:09,141 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:27:09,143 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:27:09,264 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:27:14,281 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:27:14,282 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:27:14,419 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:27:19,437 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:27:19,438 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:27:19,530 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:27:24,544 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:27:24,544 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:27:24,633 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:27:29,650 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:27:29,651 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:27:29,734 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:27:34,750 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:27:34,751 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:27:34,820 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:27:39,841 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:27:39,843 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:27:39,923 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:27:44,944 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:27:44,946 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:27:45,042 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:27:50,073 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:27:50,074 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:27:50,103 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:27:55,126 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:27:55,127 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:27:55,255 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:28:00,273 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:28:00,274 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:00,407 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:28:05,423 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) [7a22acac] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:28:05,424 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) [7a22acac] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:05,543 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) [7a22acac] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:28:10,559 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:28:10,560 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:10,637 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:28:15,656 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:28:15,657 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:15,773 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:28:20,790 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:28:20,791 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:20,868 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:28:25,895 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:28:25,897 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:25,962 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:25,996 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:28:31,022 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:28:31,024 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:31,091 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:28:36,119 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:28:36,120 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:36,235 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:28:41,253 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:28:41,254 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:41,370 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:28:46,392 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:28:46,393 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:46,514 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:28:51,531 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) [342cbfee] Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:28:51,531 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-8) [342cbfee] Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:51,633 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) [342cbfee] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:28:56,650 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:28:56,650 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:28:56,735 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:29:01,750 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:29:01,751 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:29:01,821 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:29:06,841 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:29:06,842 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:29:06,925 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:29:11,948 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:29:11,950 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:29:12,069 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:29:17,101 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:29:17,101 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:29:17,163 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:29:22,181 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:29:22,182 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:29:22,249 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:29:27,266 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:29:27,267 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:29:27,385 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:29:32,414 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:29:32,415 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:29:32,548 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:29:37,564 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:29:37,565 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:29:37,698 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:29:42,712 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:29:42,713 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:29:42,884 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:29:47,917 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:29:47,919 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:29:47,986 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:29:53,014 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:29:53,016 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:29:53,047 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:29:58,071 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:29:58,072 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:29:58,174 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:30:03,196 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:30:03,197 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:30:03,310 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-58) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:30:08,325 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:30:08,325 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:30:08,463 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:30:13,478 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:30:13,479 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:30:13,615 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:30:18,635 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:30:18,635 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:30:18,734 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_25] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_25] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_25] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_25] > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) [jsse.jar:1.7.0_25] > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) [utils.jar:] > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) [commons-httpclient.jar:] > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) [commons-httpclient.jar:] > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) [commons-httpclient.jar:] > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) [xmlrpc-client.jar:3.1.3] > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) [xmlrpc-client.jar:3.1.3] > at com.sun.proxy.$Proxy83.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_25] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25] > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:242) [vdsbroker.jar:] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_25] > at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25] > at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25] > >2014-08-31 08:30:23,750 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDSCommand(HostName = h1, HostId = 9c67d903-8424-4cda-8bb1-3d8652b92d77, vds=Host[h1,9c67d903-8424-4cda-8bb1-3d8652b92d77]) execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-08-31 08:30:23,750 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Host h1 is not responding. It will stay in Connecting state for a grace period of 160 seconds and after that an attempt to fence the host will be issued. >2014-08-31 08:30:23,844 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:631) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] >